# Study finds separate critic agents add 5.9 to 9.5 points on Terminal-Bench

> Researchers from Alibaba and Cornell compared 17 verification designs across seven frontier coding models on Terminal-Bench and SWE-bench Verified.

- Status: official (From the company, authors or authority themselves, with a link to their own page.)
- Topic: Research
- Published: 2026-11-01T17:34:56+00:00
- Updated: 2026-11-01T17:34:56+00:00
- Web page: https://ai-news.ikangai.com/story/study-finds-separate-critic-agents-add-5-9-to-9-5-points-on-terminal-bench/
- Seen in: DAIR.AI (Elvis Saravia)
- Change: research (Research)

Separate critic agents raised accuracy on Terminal-Bench by 5.9 to 9.5 points and on SWE-bench Verified by 3.9 to 6.6 points, according to a paper from researchers at Alibaba and Cornell. The cost rose to 1.13x to 1.25x baseline on Terminal-Bench and 1.00x to 1.04x on SWE-bench. The study compares 17 verification designs across seven frontier coding models and sorts them into four layers of the agent lifecycle: initial instructions, mid-turn reminders, completion gates, and independent review.

The first two layers produced modest gains. System-role instructions added 3.3 points on Terminal-Bench and 3.9 on SWE-bench. System-role reminders added 3.1 to 3.8 points, and the paper stresses that the effect depends on the model. Completion gates reached further, with larger gains at a higher price. Early user rejection gave the largest gate gains, at 6.0 and 6.2 points. Those designs cost 1.60x to 1.76x baseline on Terminal-Bench and 1.48x to 1.56x on SWE-bench. The authors recommend avoiding completion-time self-checking.

Independent review offered a better trade-off between accuracy and cost. In this design, a separate critic reviews the work, and its extra spend stays near the baseline on SWE-bench, at 1.00x to 1.04x. The paper also reports that extra review rounds or additional critics add up to 3.5 more points on Terminal-Bench.

The authors also introduce VisualBench, a benchmark for expert evaluation of visual and interactive artifacts. On it, the critic designs reach net preferences of 13.2 to 23.8 points over the baseline. The paper's key point is that reviewing the changes works better than reviewing the trajectory, meaning the record of steps the agent took.

## Why it matters

A single added reviewer step gives a measured accuracy gain at modest cost, while heavier gating can multiply spending. Teams can use these figures to justify one verification pass instead of more elaborate gates.

## What you can do

Test a separate reviewer step in your coding agent pipeline and track both accuracy and cost per task.

## Original source

- [(2610.34577) Rethinking Verification in Agent Harnesses](https://arxiv.org/abs/2610.34577)
- [Rethinking Verification in Agent Harnesses | DAIR.AI Academy | DAIR.AI Academy](https://academy.dair.ai/papers/rethinking-verification-in-agent-harnesses-2610.34577)

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "Study finds separate critic agents add 5.9 to 9.5 points on Terminal-Bench", 2026-11-01, https://ai-news.ikangai.com/story/study-finds-separate-critic-agents-add-5-9-to-9-5-points-on-terminal-bench/
