Skip to content
Predictive Systems
PSI Daily

AI Shows Its Work. New Study Says You Can Also Spot the Steps Inside the Model

Interpretability

An AI can write “first I will add, then I will divide.” A new study says those kinds of steps also show up as different patterns inside the model itself.

checking test paper

What the researchers asked

People often ask AI models to show their work. The model writes a chain of thoughts: pull out the facts, break the problem into parts, recall a formula, do the math.

Researchers at KAIST and Naver AI Lab asked something more basic. Can you tell those steps apart inside the model, not only in the text?

Their answer: yes.

How they tested it

They picked eight kinds of reasoning steps. Examples include pulling out facts, breaking a problem apart, recalling a formula, drawing a conclusion, and calculating.

They had three models solve math problems: Qwen2.5-7B, Qwen3-8B, and Gemma4-31B. They split each written answer into short pieces. They labeled each piece by step type. Then they looked at the model’s internal numbers for those pieces.

What they found

Different step types leave different internal patterns. The clearest patterns sit in the middle layers of the model.

It is not just the wording. A tool that only looked at the words did worse than a tool that looked inside. Where the step sat in the answer also did not explain the pattern.

Even tiny words like “a,” “is,” and “the” change inside the model based on the step around them. Early layers mix those words together. Middle and later layers pull them apart by step type.

Context matters too. When the team blocked the model from using the previous thirty words, the next step’s internal signal got weaker. Steps build on what came before.

Wrong answers still showed the right kind of step. A bad calculation still looked like a calculation step inside. The type was visible. The truth of the answer was not fixed by that alone.

The same idea worked on Llama-3-8B. Tools trained on one model also worked on other math and science tests. The authors say this is still mostly about math, on a small set of models. They do not say you can catch every error or guide a model while it runs.

Why you should care

Many people treat a clear written chain of thought as proof they understand the model. This study says the written steps and the internal steps often match. That is useful for people who study and monitor AI.

It also has a limit. Clear writing helps. It is not a full window into everything the model does. Other labs have found models that hide some of their hints, or that think more than they write. So: trust the steps as a clue. Do not treat them as a full confession.

Sources