DecisionRAG replaces the first-stage retriever with a hierarchical decision model
Researchers from Cornell and ASAPP propose a decision model that walks the corpus from broad topics to passages and beats dense retrieval plus an 8B reranker.
Researchers from Cornell and ASAPP have proposed DecisionRAG, a retrieval method that drops the first-stage retriever altogether. It replaces that retriever with a decision model that walks a document collection from broad topics down to documents and passages. Without any task-specific training, the authors report that it beats dense retrieval followed by an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions. The motivation is what the authors call a capability asymmetry. If the first-stage retriever misses a passage, even a strong generator downstream never sees it. Most retrievers still depend on lexical matching or similarity in fixed embedding spaces. The mechanism works level by level. At each level, a decision model such as Jev scores every candidate branch in a single forward pass, without generating any tokens, and then keeps the most promising branches to explore further. The paper says that under a balanced hierarchy, the number of sequential decisions grows logarithmically with corpus size. The authors also argue that judging candidates together, in a shared context, makes their relevance estimates comparable. That design choice accounts for 7.7 R@1 points on MultiDoc2Dial and 6.5 Hit@1 points on Natural Questions over scoring each candidate separately. The evaluation covers four benchmarks, including a corpus of 2.68 million Wikipedia paragraphs. Retrieval takes 0.7 to 1.8 seconds per query and costs 0.2 to 2.3 cents per query. The 13.8 and 17.4 point gains are the zero-training results, and the comparison is against dense retrieval plus an 8B reranker. A separate figure concerns a Qwen3-8B judge trained on in-domain data, which reaches 60.7 R@1 on MultiDoc2Dial. That is 5.7 points above the best compared published task-trained retriever. The method has a structural weakness that follows from how it searches. Because the walk prunes branches at each level, a branch dropped early cannot be recovered later in the same pass. The method depends on a well-organized corpus hierarchy. A pruned branch stays lost unless the search budgets grow or dense retrieval adds candidates. Its results, in other words, rest on how well that hierarchy reflects the corpus.
What changed
DecisionRAG replaces the first-stage retriever with a hierarchical decision model that walks the corpus from broad topics down to passages.
Type: Research
Why it matters
Retrieval pipelines built on keyword matching or fixed embeddings may miss content that a hierarchical decision model could reach, but only where the corpus is well organized.
What you can do
Check whether your document corpus has a clear topical hierarchy before considering decision-based retrieval.
Original source
Where we saw it
DAIR.AI (Elvis Saravia)
AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works