ARCHER: Adaptive Recovery Routing with Conformal-risk Calibration for Coding Agents
Abstract
Coding agents increasingly operate in executable environments where a failed attempt produces actionable feedback rather than merely an incorrect answer. Existing cost-aware systems typically treat such failures as cascade decisions: try a cheap model first, then escalate hard cases to a stronger and more expensive model. In coding, however, execution feedback can make additional cheap-model recovery worthwhile, turning post-failure inference into a budgeted deployment problem: when should an agent spend more cheap compute, and when should it escalate? We formulate this decision as recovery routing and train a supervised router from execution rollouts. To make the same router usable under changing budgets, we add a Conformal Risk Control (CRC) layer that maps a target budget to a deployment-time cost penalty, allowing the same trained router to adapt to different deployment budgets without retraining while providing marginal expected-cost control. Across held-out failures from five coding benchmarks, cheap recovery and escalation solve complementary subsets of instances, so no single fixed policy is uniformly best. The resulting calibrated frontier improves over fixed actions, prompt-only routers, and a binary cascade baseline. In the main GPT-5.4-nano/GPT-5.4 setting, the frontier includes a calibrated operating point that exceeds always-escalate's solve rate at 35% of its mean recovery cost. Code is available at https://github.com/Qijia-He/ARCHER.