Drew Breunig bakes a working NES emulator into Qwen3.5-4B's weights
Drew Breunig says he embedded a working emulator for the NES 6502 CPU in the weights of Qwen3.5-4B, which can run NES ROMs such as Super Mario Bros very slowly.
Drew Breunig has published nesformer, a checkpoint whose weights implement a working emulator of the Nintendo Entertainment System's 6502 CPU. It can run NES ROMs such as Super Mario Bros. (World 1-1), though very slowly. Simon Willison linked to the project on 30 October 2026. The checkpoint loads as a stock Qwen3_5ForCausalLM model with no custom code. The README says nobody trained its weights, because a compiler computed every one.
The mechanism is the unusual part. According to the README, the 6502 core, the 2 KB of work RAM and a cartridge's program ROM all run inside the model, which predicts the CPU's execution log one token at a time. A small host program handles the rest of the console: it draws the picture, reads the controller and fires the vblank interrupt. The PPU and APU are not in the weights, so video and sound generation sit outside the model.
The memory is built from Gated DeltaNet state. Each head keeps a 128 by 128 matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row, and a one-hot query reads that row back. Each row therefore acts as a memory cell, and the cartridge, RAM and registers all live in rows. The README credits Divyansh Agrawal's Qwenputer, which compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B, for both the compiler and this trick.
The phrase 'weights of Qwen3.5-4B' needs a qualifier. The checkpoint uses Qwen3.5-4B's DeltaNet head shapes and MLP width, but its hidden size is 768. It has 466.6M parameters in bf16 (933 MB), and only 9.3M of those are nonzero, which is why the weights zip to 3.3 MB. Seven DeltaNet layers hold the circuit, and one all-zero attention layer exists because Hugging Face's cache needs it. So it is a model with Qwen's architecture, filled with compiled values, not the 4B model with altered weights.
Speed is the obvious cost. The README says a frame takes 1 to 6 seconds, so 600 frames, which is 10 seconds of NES time, takes around 25 minutes. The default numba engine runs at about 1,650 tokens per second on one CPU core of the build machine, the numpy engine at about 210, and stock transformers at about 10. A real NES would need about 700,000 tokens per second. An idle-loop skip cuts the token count by 2.6 to 4.2 times. Overall, the README puts nesformer at 150 to 290 times slower than the console.
The checks compare the model with the project's own reference CPU. The reference matches nestest's golden log for 3,348 instructions and the py65 emulator on 5.7 million instructions of game code. Free-running, the weights reproduced the reference trace for Thwaite (700,000 tokens, 271 frames) and Full Game (2,645,885 tokens, 420 frames, with all 210 saved frames matching). Those frame matches test the host's rendering against the same reference. They show that the model reproduces the reference emulator's output, not that the PPU matches hardware.
The limitations are substantial. Only NROM cartridges (iNES mapper 0) are supported, because the CPU can address just 32 KB of program ROM at a time. Larger games use bank-switching chips such as MMC1 or MMC3, and the README says nesformer 'can't follow a bank switch yet.' Only official opcodes work, and unofficial ones emit HALT. There are no IRQs from the APU frame counter or DMC, and there is no sound. Mid-scanline effects are not reproduced, and sprite-0 hit and sprite overflow are approximate.
In a Hacker News comment, Breunig said he used Opus 5.5 to build the system and shared the prompts. They asked whether the chip and RAM could be emulated the way FPGA projects do, what model size that would need, and whether smaller, faster models could reach the standard clock speed. In a follow-up comment, Breunig wrote 'I think we should build it! Just for fun. It's okay if it's not fast.' Willison reports that Opus found and then used a set of NES CPU conformance tests released 15 years ago by Christopher Pow.
What changed
nesformer is a checkpoint with Qwen3.5-4B's architecture whose weights implement a working NES 6502 CPU emulator that can run NROM cartridges very slowly.
Type: Research · Applies to: nesformer, Qwen3.5-4B
Why it matters
The project shows that fine-tuning can encode deterministic program execution into a model's weights. The demo runs very slowly, so it is a research curiosity rather than a practical way to run software today.
Original source
Where we saw it
Simon Willison
AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works