Skip to content
Predictive Systems
PSI Daily

Is AI Really Readable Inside? Even Its Champion Says Not Yet

InterpretabilityAllan C. Tan, MS

The researcher behind a viral claim that AI is tidy inside admits it is not everything solved. Experts call it a hunch until his paper is out.

Tangled cables beside neatly labeled strands, with the overlay 'Is AI Really Readable Inside?'

The researcher behind a viral claim that AI is far easier to read than we thought is now walking part of it back. In an exclusive interview with MIT Technology Review, Goodfire's Nick Cammarata said, "It's not infinitely good news; it's not everything solved." Top experts agree the jury is still out.

Cammarata's essay, "Neural Networks Are Already Clean," argues that most parts inside AI models each do one clear job. We broke down the essay in plain English here. The interview adds the backstory. Cammarata says he spent "10 hours a day, seven days a week" for most of a year looking at neurons, and he was surprised by how strongly people reacted.

The Experts Split

Stanford's Chris Potts told MIT Technology Review the essay is little more than a hunch. "Even a large number here is not a substitute for rigorous evaluation," he said. Jacob Hilton of the Alignment Research Center wrote on X that researchers understand less than 0.01% of how top models work, and that the essay gives the public "an unrealistic picture." Google DeepMind's Neel Nanda called it "interesting work" but not "a significant update to the big picture," and sees it as a useful tool rather than a breakthrough.

What Would Settle It

The essay is not peer reviewed, and its technical paper is not out yet. Goodfire also sells interpretability tools, so it has a stake in the answer. The real test is whether outsiders can repeat the results on today's large models, and whether reading single neurons explains how they work together. Cammarata himself admits that question is still open.

For anyone who relies on AI, the takeaway is simple. Reading an AI's mind looks a little more possible than it did last month, but nobody has proven it yet.

Sources

  1. MIT Technology Review, "We were looking at it wrong": The explosive claim that's upended the AI world, October 19, 2026
  2. Goodfire, Neural Networks Are Already Clean (essay, not peer reviewed), October 15, 2026
  3. The Decoder, Is AI interpretability close to a breakthrough? A new essay has researchers divided, October 16, 2026
  4. Hacker News, Discussion: Neural Networks Are Already Clean
  5. Predictive Systems, AI's Black Box May Be Tidier Than Anyone Thought, October 18, 2026

Image: AI generated illustration, with text overlay added.