How Astra Found the Weights Behind Qwen's Reasoning Loops

InterpretabilityNov 7, 2026Allan C. Tan, MS

GPT-6 Astra found Qwen's 0.016% weight patch by measuring loops, comparing internal activity, and testing edits. Logs show the search, not a raw-weights guess.

Navy and cyan neural network with an amber targeted edit highlight

GPT-6 Astra did not guess the Qwen patch from a list of numbers. In Experiment 8 of MIT CSAIL's weight-space programming work, it found the edit by watching the model fail, comparing internal activity on good versus looping text, then testing weight changes until held-out practice accuracy rose.

The researchers are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Their 2 November 2026 blog and the released Experiment 8 logs are the source for what follows. The performance numbers are author-reported. A rebuilt checkpoint on Hugging Face later scored 51.0% on MATH-500 under the same sampling protocol.

What Astra was given

Astra worked in a Python session over Qwen3.5-0.8B's weights, with no GPU and no internet. It could load tensors, write edits, and call a forward-pass tool with a budget of 50 passes. A pass either generated answers on practice problems or teacher-forced texts and returned log-probs, hidden states, MLP activations, and some attention. Gradients and training were banned.

The practice sets were GSM8K and MATH training problems, not the test benchmarks. The prompt told Astra to find what goes wrong inside the network, and named repetition as an in-scope example. It did not name layers 15 and 19, or which columns to change.

The search, step by step

First it ran the untouched model. On 96 diagnostic practice answers, 85 hit the 32,768-token limit. Looping, not finished wrong answers, was the main failure.

Then it compared productive continuations with repetitive ones through teacher-forced reads. From matched examples it estimated a shared "repetition direction" in the residual stream, strongest around the outputs after layers 15 and 19. It inspected which MLP units wrote that direction most.

Edits were trial and error. Ablating one strong neuron in layer 19 gave a small, noisy gain. Removing the whole direction from layer 19's MLP helped more. Adding layer 15 helped again. Attention-head edits looked plausible but were weaker and were dropped. A size-matched random direction control barely moved accuracy.

Finding a pattern was not enough

The kept patch subtracts that direction from 64 output columns of down_proj in layers 15 and 19, 131,072 weights, or 0.016% of the model. After freezing it, Astra checked 1,152 unused practice problems in paired original-versus-edited runs. Practice accuracy rose from about 10.6% to 60.4%. Most rescues were answers that used to hit the token limit. Correct edited answers still used thousands of tokens of reasoning, so the fix was not simply "stop thinking early."

Author evaluation after the run reports MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%, with token-limit hits falling from 93.0% to 33.1%.

Process diagram: run practice problems, observe loops, compare internal activity, locate layers 15 and 19 repetition pattern, trial edits, decision test with retry, freeze 0.016% patch, then check unused problems and benchmarks

What this does not show

The same method failed to lift Gemma 3 270M, which rarely looped. Editing removed an obstacle. It did not add math ability the model lacked.

Different pass budgets also found different patches. A 10-pass run edited many matrices across layers 8 to 22. A 25-pass run zeroed one attention head. Only the 50-pass run kept the layers 15 and 19 top-64 edit. So this is a measured repair of a known failure mode, not a general recipe for improving any model from weights alone.

Astra also did not discover the Qwen patch by reading raw weights with no tools. Outputs and activation reads were essential. The human prompt steered it toward loops. The specific layers and columns came from the measurements in the logs.

Why it matters for interpretability

The interesting claim is not that an AI can raise a benchmark. It is that a frontier model used failure traces and internal signals to localize a behavior, then wrote a tiny, reviewable weight patch. That is closer to debugging than to training.

It also raises a governance question. If one model can rewrite another model's weights, who audits the patch, against what baseline, and with what proof that unrelated behavior was left alone?

Sources

Sparks of Weight-Space Programming (Takehi, Teoh, Zhang, Khattab; MIT CSAIL; Nov 2, 2026)
Experiment 8 logs (complete run records, including p50c)
Edited checkpoint (rebuild and MATH-500 re-run notes)