# MIT CSAIL team uses GPT-6 Astra to edit Qwen weights and raise math score > The MIT CSAIL team reports in a paper that GPT-6 Astra edited two layers of Qwen3.5-0.8B and raised its MATH-500 accuracy from 7.9% to 48.1%. - Status: reported (One outlet, not independently verified yet.) - Topic: Research - Published: 2026-11-02T23:04:05+00:00 - Updated: 2026-11-04T18:54:05.666939+00:00 - Web page: https://ai-news.ikangai.com/story/mit-csail-team-uses-gpt-6-astra-to-edit-qwen-weights-and-raise-math-score/ - Seen in: Turing Post, Latent Space, Forward Future (Matthew Berman) Four MIT CSAIL researchers, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, published a paper on November 2, 2026. It argues that frontier language models can read and write the numerical weights of neural networks directly, without an optimizer or gradient descent. The paper's own summary calls these early results and says the approach "might be a big deal" for optimization and interpretability. Weights are the numbers that training adjusts. The authors call the approach Weight-Space Programming. The model reasons about what each part of a network computes, or should compute, then changes the numbers to match, with no training loop. The first test was a reading task. Astra received the 17,253 weights of a small policy network, written as text, plus a description of a 10×10 grid and five actions, but no game name. The network came in four versions, trained with PPO on Breakout, Pong, Pac-Man and Freeway, all with identical shapes, so the only difference was in the numbers. Astra named the right game for all four. For Breakout, it noted that the agent's input weights concentrate along the bottom row, which it read as a paddle, and it reported 99% confidence. The reading experiments also have limits. For a diffusion model that generates circles, squares and triangles, Astra found the shape in the output layer, but it drew every shape as an outline, although the models generate filled shapes. The paper reads that error as a sign that Astra was reading the geometry of the weights rather than running the network. In a harder test, the team removed the input and output layers from three 7.16-million-weight character-level transformers and gave Astra only the four middle blocks. It was not allowed to run the network on any input. Astra named the training language correctly in all three cases and identified every copying head. The paper notes, though, that it reached those heads with a method by Elhage et al. that the team never mentioned to it. Writing networks is a separate claim. In the MinAtar games, Astra wrote networks from scratch in one reply, with no data, loss or gradients. Its scores were 517.4 on Breakout, 53.1 on Seaquest and 53.2 on Asterix. The comparison points are PPO networks trained for 100 million steps (15.8, 5.6 and 25.8) and for 500 million steps (25.3, 88.9 and 55.2). Astra beat the 500-million-step PPO network only on Breakout, where the paper reports a figure about 20 times the PPO score. On the other two games it came close or fell short. With thinking mode on, Qwen repeated steps it had already written, and the paper notes that Qwen's own model card warns about such loops. Astra compared the model's internal signals during loops with those during steps that made progress. It found that layers 15 and 19 reinforced a pattern tied to repetition. Its edit subtracted that direction from 64 output columns of the down_proj matrix in each of those layers, touching 131,072 weights, or 0.016% of the model. The team gave Astra a budget of 50 forward passes on practice problems from the GSM8K and MATH training sets, and the test questions stayed private. The reported numbers move together. MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%. The share of answers that hit the 32,768-token limit fell from 93.0% to 33.1%. Coauthor Alex Zhang says the team initially thought Astra had edited a new math skill into Qwen. The paper also tested Gemma 3 270M. That model generally finished its answers but made arithmetic and algebra errors, and the paper reports no clear gain there. The prompt is part of the story. It told Astra that if Qwen "repeats itself and never finishes, finding and fixing what drives that repetition counts." Teoh said the team gave its short assignment to Claude Code, which added the repetition hint while expanding the prompt, without further human feedback. The authors also ran a shorter prompt without the repetition example. At 25 passes it reached 14.1%, the same score as the run with the hint, and the table has no 50-pass result for that shorter prompt. The checkpoint is published on Hugging Face. A page for it says the edit was rebuilt and hash-verified. A separate run on full MATH-500, with 500 problems and four samples each, scored 51.0%, and 31.6% of answers hit the token limit. That page also says a 25-problem smoke test of the base model scored 5.0%. The re-run differs from the paper in one respect: the paper evaluated a text-only export, while the checkpoint is the full multimodal release. The page says the language-model tensors are identical. ## Why it matters Automated model editing could become a practical way to diagnose and patch specific failure modes, though the gains here come mostly from fewer unfinished answers. ## What you can do Teams experimenting with model editing should check whether gains hold on private tests and separate fixes for finishing answers from real capability gains. ## How the story developed - 2026-11-04: Commentators argue the 48.1% gain mostly reflects removing a looping failure mode from a pathological 7.9% baseline, and say the edit reportedly failed on a different model. - 2026-11-04: Adds that the change touched about 0.016% of weights and that the gains still need independent reproduction. - 2026-11-04: The video adds detail on the paper's experiments, including identifying games and diffusion outputs from weights and writing networks from scratch. ## Original source - [Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights](https://rikiyat.github.io/blog/weight-space-programming/) - [GitHub - RikiyaT/lm_understand-logs: Complete logs of the experiments in the blog post 'LLMs can understand and write neural network weights' · GitHub](https://github.com/RikiyaT/lm_understand-logs) - [RikiyaT/Qwen3.5-0.8B-weight-space-programming · Hugging Face](https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming) - [Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights](https://www.reddit.com/r/LocalLLaMA/comments/1xm6jow/sparks_of_weightspace_programming_llms_can/) - [MIT Researchers Discover AI Can Rewrite Its Own Weights](https://www.youtube.com/watch?v=j09uV1oAuIo) ## Further reporting - [→ Read the full article here](https://venturebeat.com/technology/like-gene-editing-for-neural-networks-mit-researchers-used-gpt-6-astra-to-edit-qwens-open-weights-directly-boosting-math-performance) --- AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it.