AgentBaton detector cuts API cost 65.9% by handing off to cheaper models mid-task
Researchers from NVIDIA, HKUST and UIUC report a one-time handoff detector that cuts API cost 65.9% on average while largely preserving accuracy.
Researchers from NVIDIA, HKUST and UIUC report that a single handoff from an expensive, capable model to a cheaper one cuts API cost by 65.9% on average across 11 model pairs. The handoff is timed by a learned detector, and the authors say it largely preserves task accuracy. The paper, posted to arXiv on 25 October 2026, is titled AgentBaton and targets long-horizon agent tasks, where an agent takes many steps against a tool or environment before finishing. The authors argue that reliable performance on such tasks often comes from expensive capable models offered by cloud providers. Cheaper lower-tier cloud models and smaller open-weights models that run on local hardware are also becoming available. Their question is whether a capable model and an efficient model can share one task instead of splitting work by query, as traditional query-level routing does.
The authors tested handoffs in both directions and found them asymmetric. Efficient-to-capable handoffs generally underperformed capable-only execution and produced little or no cost reduction. Capable-to-efficient handoffs made at suitable stages of task progress largely preserved or even improved accuracy while cutting cost substantially. From this, the authors concluded that one well-timed handoff is good enough, which reduces the design problem to choosing when to switch. AgentBaton is a lightweight detector that learns the switch point from recurring patterns in task progress. A DAIR.AI summary reports that it runs at a 338 ms median latency on one H100 for inputs of 2K to 121K tokens.
The headline figures come from the abstract and the paper's summary: a 65.9% average API cost reduction across 11 model pairs and two benchmarks, with accuracy gains of up to 8.8 percentage points over capable-only execution. The benchmarks are DeepSWE v1.1 and Terminal-Bench 2.1. The word largely matters: the abstract says accuracy is largely preserved, not fully preserved, so the claim is not that accuracy is unchanged. The cost accounting deserves the same care. The summary notes that the cost figures assign zero cost to locally deployable models. Savings therefore look larger when a cheap model runs on hardware the team already owns, since no per-call fee is counted for it.
The abstract also says AgentBaton extends the empirical accuracy-cost Pareto frontier across various reasoning-effort settings. In those settings, it reaches accuracy-cost trade-offs the frontier did not previously offer. The approach depends on handing off at a suitable stage of task progress, where the cheaper model can continue without giving up much accuracy. The paper's evidence concerns coding and terminal tasks, not every kind of agent work.
What changed
A one-time handoff from a capable model to a cheaper one, timed by a learned detector, is reported to cut API cost 65.9% on average while largely preserving accuracy.
Type: Research
Why it matters
Routing part of an agent's work to cheaper models could lower spend for teams running long agent tasks. The savings depend on which model pairs are used and on whether the cheaper model runs locally.
What you can do
Measure which steps of your long agent runs could move to a cheaper model before committing to a routing design.
Original source
Where we saw it
DAIR.AI (Elvis Saravia)
AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works