# FrontierML benchmark tests whether AI agents can turn extra compute into better models

> FrontierML gives agents $10, $50 or $200 to spend on 21 ML tasks, and the authors report that stronger agents reliably improve with more money.

- Status: official (From the company, authors or authority themselves, with a link to their own page.)
- Topic: Research
- Published: 2026-11-16T14:01:15+00:00
- Updated: 2026-11-16T14:09:24.540416+00:00
- Web page: https://ai-news.ikangai.com/story/frontierml-benchmark-tests-whether-ai-agents-can-turn-extra-compute-into-better/
- Seen in: Import AI (Jack Clark)
- Change: research (Research)

Agents built on frontier models can turn extra spending into better machine learning models, and stronger agents improve more reliably as the budget grows, according to a new benchmark called FrontierML. Its authors, from Induction Labs, give each agent a 12-hour CPU sandbox, a wallet of $10, $50 or $200, a few validation samples and a task description. They then score the submitted models on 21 tasks spanning LLM training and efficiency, vision, audio, computer use, applied NLP, and math and games.

The main measure is the change in mean score across the 21 tasks as the wallet grows. A human researcher gained 56 points at $10 and 66.1 at $200. The best system the authors tested, Claude Opus 5.5, gained 45.4 at $10 and 58.8 at $200. The top open-weight model, Kimi K3.1 Pro, gained 25.5 and 29.6. The gap between the best agent and the human is widest at the small budget and narrows as the wallet grows, though the human still leads at both levels.

The wallet is spent through Spin, an open-source command-line tool and service that lets agents buy on-demand GPUs from 17 compute providers and metered tokens from OpenRouter. The authors fix each GPU's price at a snapshot taken on September 29, 2026, so the same budget buys the same compute in every run. That matters for comparability, because cloud prices move. The tasks give agents little direction. According to the authors, agents may choose architectures, source data, provision compute and optimise inference. In 28% of trajectories, agents add newly trained parts to pretrained models or train a new model from scratch. A separate overfitting test gives the agent only a few validation samples, while the score comes from a large out-of-distribution held-out set served from an air-gapped H100 machine.

The spending figures suggest agents leave much of their money unspent. The authors write that agents spend 45% of their wallet on average, even when told there is no penalty for using all of it, and no individual agent spends more than 80% at any budget. Agents also finish early, with frontier models averaging about half of the 12 hours. The benchmark's own analysis points to weak research before spending. Agents mostly chose models for synthetic data generation from memory, and over 67% of their OpenRouter calls for synthetic data went to models more than a year old. Across all runs, only two web searches were made about which GPU to use. The authors present these results as evidence of inefficient spending. That is a reasonable reading of the data, but the post does not say what efficient spending would look like or whether a higher score would follow from it.

The benchmark also shows limits that the announcement does not dwell on. The authors say some tasks are out of reach at these budgets, which is why they are building tasks with budgets from $1,000 to $100,000. Each agent was also run at only one reasoning effort. Scores are averaged across 21 tasks, so a mean gain can hide large differences between tasks. The post shows one task, screenshot generation, where approach families scored very differently. The 28% figure describes how often agents built new parts, not how often those parts helped, so it does not by itself show that custom architectures produce better models.

## Why it matters

Benchmarks like this show whether agents can use budget to improve their own training, which bears on how much autonomous AI R&D companies may delegate. The gap between the best closed model and the best open-weight model is also useful to buyers weighing open models.

## Original source

- [FrontierML: a challenging, end-to-end machine learning benchmark · FrontierML](https://www.frontierml.org/blog/)
- [GitHub - induction-labs/frontier-ml: Given more money, can an agent train a better model? · GitHub](https://github.com/induction-labs/frontier-ml)

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "FrontierML benchmark tests whether AI agents can turn extra compute into better models", 2026-11-16, https://ai-news.ikangai.com/story/frontierml-benchmark-tests-whether-ai-agents-can-turn-extra-compute-into-better/
