# Microsoft Research's LongCapHarness boosts long-context scores without training

> Microsoft Research's LongCapHarness changes the code around a frozen model, splitting tasks into smaller operations, and reports large gains on long-context tests.

- Status: reported (One outlet, not independently verified yet.)
- Topic: Research
- Published: 2026-11-20T12:01:55+00:00
- Updated: 2026-11-20T12:08:36.333528+00:00
- Web page: https://ai-news.ikangai.com/story/microsoft-research-s-longcapharness-boosts-long-context-scores-without-training/
- Seen in: AI Engineering
- Change: research (Research)
- Applies to: LongCapHarness, Llama3-8B, Qwen2.5-3B, GPT-5.6-Sol

## What changed

Microsoft Research's LongCapHarness changes the code around a frozen model instead of its weights, splitting tasks into smaller operations, and raises Llama3-8B's aggregate HELMET and RULER score from 15.14 to 79.32.

Microsoft Research says it can raise long-context performance without changing a model's weights. Instead, it changes the code around the model. The method, LongCapHarness, builds agent workflows that split a task into smaller operations, with some steps handled by the model and others by ordinary code. On the aggregate of HELMET and RULER, Llama3-8B's score rose from 15.14 to 79.32.

One experiment concerns passage ranking. One approach asks the model to give each retrieved passage a relevance score, then sorts those scores in code. That only works if a score means the same thing across separate calls. The researchers instead had the model compare passages within small groups, and code combined those group rankings into one complete order. For Qwen2.5-3B, the mean ranking score rose from 40.50 to 56.28, with the model's weights unchanged.

LongCapHarness defines operations with set inputs and outputs. Some operations call the model, and others run as code. Intermediate results are kept outside the model's context and passed to later steps when needed. In the ranking case, the model decides which passage in a group better answers the question. The program keeps passage IDs attached and assembles the final order. Splitting text into chunks would not produce this structure by itself, because each call still has to be specified: what it receives, how it expresses its judgment, and how its output feeds the next step.

The researchers used GPT-5.6-Sol to develop the harness against examples with reference answers. The optimizer revised the operations, the state representation, and the control logic. The workflows were then fixed before being tested on unseen inputs, so the reported gains did not come from continued revision during evaluation. The development process still depends on examples and feedback, and the "no training" label refers only to the model's weights.

The gains were uneven. The largest improvements came from explicit operations such as lookup and counting, while gains on question answering were less consistent. The write-up also says the method helped Qwen much more than Llama. It does not say how many inputs the unseen test set contained.

## Why it matters

Teams working on long-context retrieval may be able to fix some failures through workflow design before committing to fine-tuning. The gains depend on the model and task.

## What you can do

Before planning a fine-tuning run, test whether your current model makes individual judgments reliably, for example by comparing pointwise scoring with group comparisons on the same labeled queries.

---

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.

Cite as: IKANGAI AI News, "Microsoft Research's LongCapHarness boosts long-context scores without training", 2026-11-20, https://ai-news.ikangai.com/story/microsoft-research-s-longcapharness-boosts-long-context-scores-without-training/
