AI News
All stories
Reported

Microsoft engineers merged 1,211 pull requests in a day using scoped AI agents

Nine Microsoft engineers used written specs and single-task agents to merge 1,211 pull requests in a day and move a live runtime from Python to Rust.

AI in Practice · · 4 min read

Nine Microsoft engineers merged 1,211 pull requests into the main branch of the Copilot Cowork backend on October 21. Bas Brekelmans, a partner software engineer on the team, reported this in a Microsoft blog post. The post says the team went from an empty repository to worldwide general availability in about four and a half months, then moved the live runtime from Python to Rust. Copilot Cowork is an agentic execution layer that performs multi-step workflows on a user's behalf. The first commit came on January 13. The team launched to its first enterprise customers in the Frontier program on March 30 and reached general availability on June 16.

The first lesson is that no code gets written without a markdown spec. Brekelmans says the spec covers what the change does, how the team will know it works, and what it must not break. The team found that agents were rarely the thing slowing it down. The real bottleneck was not knowing exactly what it wanted. A vague request produced code that looked reasonable, survived a quick read, and then did the wrong thing. Fixing that cost more time than a proper request would have. Many specs start as speech: a person talks through a feature, an agent drafts the spec, and the human edits it. Brekelmans wrote that this did more for the team's speed than any single model upgrade.

The second lesson is scope. Each agent works in its own copy of the repository on one task from one spec. Merged pull requests rose from 257 in January to 2,754 in May, and the busiest week before general availability had 783. The post credits narrow tasks with fewer mistakes. It also says the team chose models per task as a quality-and-cost tradeoff. The model runs in a container with no user credentials and no direct network path to Microsoft 365. A trusted host outside the container handles identity, policy, approvals, tools and audit. Brekelmans writes that the model can plan and ask for anything, but the host decides what actually happens. Because the agent talks to the host only through a defined interface, the team could replace the entire host during the Rust migration without changing the agent.

The post is specific about review failures. Several engineers built their own review bots. The team says each reviewer picked up rules from its own misses, and each one went through at least 10,000 generations. A useful first review usually arrives within several minutes, with a median of about four and a half minutes on a sample of recent merged pull requests. The post also admits that reviewer agents miss things. In one sensitive change, a group of them found no problem, while people found four blocking issues in the same diff. Findings are split into blocking and advisory tiers.

The post tells of a check that could never fail. It concerns a size limit on instruction files. The limit was exceeded, yet the check stayed green the whole time. The script piped its output through tail, and a shell pipeline returns the exit code of its last command. The team found similar cases, including a formatting check handed an empty file list. Its resulting rules require every check to report how many files it examined. Every new check must also come with a test that makes it fail on purpose. Brekelmans argues this matters more with agents, because agents keep working until checks pass. A check that passes without testing anything will eventually be found.

The Rust work also reshaped the codebase. Agents had put much of the first Rust code into the biggest crate, and the main runtime binary reached about 290,000 lines. The team's August audit found 86 crates with a dependency chain 13 levels deep. The team adopted a rule: each crate is a contract, a kit, an implementation or a binary, and implementations may not depend on each other. By October 7 the number of crates was about 176, and the maximum allowed dependency depth had fallen from 10 to three. The post is candid that build-time gains were smaller than predicted, since the compile cache already hit about 99% of the time on pull request builds. The measurable gain it reports is structural. The longest chain of crates rebuilt after a base-crate change fell from 11 to three.

The reasons for the migration are in the post's Python measurements. Each Python process has one event loop. Under load, the team measured stalls of up to 11.9 seconds. During a stall, everything in that process waited, including lock renewals, so some turns lost their locks partway through. The team found that 16.7% of session setups took exactly 5.000 seconds. That turned out to be a timeout firing, not real work. When memory ran out, the killed process was almost always the Python one, 539 times in one week on a single cluster. Moving blocking work off the loop helped, which the team took as evidence that the single loop was the real limit. Rust commits rose from about 200 in July to roughly 3,900 in September. The cutover ran from September 23 to October 10.

The post's own caveat is worth keeping alongside its results. It says its six practices are not a recipe for matching its pull request count.

What changed

The Copilot Cowork backend runtime was moved from Python to Rust, with the cutover running from September 23 to October 10.

Type: Other · Applies to: Copilot Cowork backend

Why it matters

Teams scaling coding agents can see concrete practices, a written spec per change and one task per agent, that the source links to high output.

What you can do

Require a written spec for each agent-made change and limit each agent to a single task before scaling output.

Original source

Where we saw it

1 source

Demetrios from the AAIF

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works