Goodfire Says a Model's Own Neurons Are Readable

Goodfire says you can study a large language model by reading its own neurons, rather than training a second model to decode them. In Qwen3.6-35B-A3B, switching off the weakest 92.2% of neuron signals in use for each word raised the model's error only 2.9%. These are company figures and have not been peer reviewed.

Why researchers stopped looking at neurons

For years, single neurons looked too mixed to study. One unit seemed to react to unrelated ideas at the same time. The usual fix was a sparse autoencoder, an extra model trained on the original network's internal numbers. It rewrites those mixed signals into a larger set of mostly silent "features," each meant to stand for one concept. Researchers then studied the features, not the model itself. Goodfire's new writeup follows an October essay that argued this extra step may not be needed.

What the new numbers show

The test kept only the 320 strongest of 4,096 neuron signals in each layer, for each word the model processes. Reasoning got worse at first. The stripped-down model looped and second-guessed itself. A short fine-tune, a small fraction of the original training, then recovered much of the lost performance. In a sample of 2,734 neurons, 69% had one main job. Given short written descriptions, a judge model could tell which text made a neuron fire 95% of the time, where a coin flip would get 50%.

What this does not settle

The results cover one open model, tested by a company that sells interpretability tools. Nobody has repeated the work independently. Interest has been light, with 2 points and no comments on Hacker News as of 11 November. The 95% figure comes from a two-choice test, not proof that every firing is understood. Weak signals were set aside, and the authors say they still need to show the findings hold on other models. Goodfire has published a paper and a neuron atlas with detailed pages for those 2,734 units. The real test is whether outsiders can redo the work, and whether reading single neurons explains how they work together.

Sources

1. Neurons as an Interpretable Basis (Goodfire research post, 10 November 2026)
2. Neurons as an Interpretable Basis (PDF) (Goodfire)
3. Neuron Atlas for Qwen3.6 (Goodfire)
4. Neural Networks Are Already Clean (Goodfire essay, 15 October 2026)
5. Neurons as an Interpretable Basis (Hacker News, 2 points and no comments as of 11 November 2026)

Photo: "Mouse cerebellar slices showing Purkinje neurons" by Achinraj, via Wikimedia Commons, CC BY 4.0. Graded and overlaid by PSI.