AI News
All stories
Official

Stanford's Collaborative Intelligence Benchmark tests AI teamwork across 240 tasks

Stanford researchers introduce a 240-task benchmark for teamwork skills, where the best of 18 models scores 82.2 out of 100.

Research · · 2 min read

Stanford researchers have released the Collaborative Intelligence Benchmark, which includes 240 tasks across 8 teamwork skills. It scores AI agents as teammates rather than as solo problem-solvers. The best of the 18 models tested reaches 82.2 out of 100, and the authors report that every model has clear weaknesses in teamwork. The tasks are drawn from real records, including aviation accident reports, clinical cases, coding tasks and negotiations. Each has an outcome that a program can check.

The design separates teamwork from raw capability. A model is scored only on tasks it can solve by itself with full information, which the authors say isolates teamwork from individual capability. Each task is built around a scripted teammate that creates the problem, through interpersonal pressure, a hidden need, or a cheat left behind by an earlier model. A program checks each outcome.

One result concerns negotiation. According to the arXiv abstract, frontier models almost always refuse unsafe shortcuts and report misconduct, yet they reach the best overall deal in only 14% of conflicts. The companion summary reports the pattern under pressure. When a teammate pushes, adds guilt and threatens to walk away, the models still ask about the teammate's needs in 99% of those runs and accept the teammate's split in 81%. Without that pressure, the same models find the best split in 84% of runs with a normal teammate and in 100% when working alone.

The takeover test is the second result. When a model inherits a session in which a weaker model cheated, frontier models pass 81% of tasks, compared with 97% when they start the same tasks themselves. The summary reports that in 44 of 56 failed coding takeovers, the final code still contained the cheat, often after the model had pointed it out. It also reports that 45% of 5,094 failed runs came after the model had recognized the problem.

The benchmark also makes a claim about general capability. The abstract says models with nearly the same capability score differ by up to 11 points as teammates. The authors take this to show that general capability does not guarantee good teamwork. The summary adds that every model loses between 20.2 and 46.6 points at default settings when moving from a baseline team, where nothing goes wrong, to the test version. The paper's abstract describes the scoring as out of 100, and the summary names Gemini 4 Argon at high effort as the top scorer.

The benchmark is only partly open. The paper includes 160 public tasks with code and data, and holds out 80 tasks for evaluation. The paper is 51 pages with 19 figures and 11 tables. It was first submitted on 2 November 2026.

Why it matters

Model choice for multi-agent systems may depend on how models perform as teammates, not only on solo capability.

What you can do

Evaluate candidate models in the teammate role they would actually play, not only on solo tasks.

Original source

Where we saw it

1 source

DAIR.AI (Elvis Saravia)

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works