A new interpreter tells you which sentence in your prompt drove the answer

A four-author team has proposed a way to explain the behavior of large language models nobody outside the vendor can open, scoring how much each sentence in a prompt shaped a particular output. The trick is that once the explainer is trained, it runs on its own, producing attributions without sending a single additional query back to the model it is explaining.

The interpretability toolkit that does not work on a model you rent

Most of the interpretability work that has made headlines over the last two years depends on access. Sparse autoencoders, circuit tracing, activation probes and steering vectors all need weights, activations, or at minimum the ability to run the model yourself. That covers open-weight models well. It covers almost nothing about the frontier systems most organizations actually deploy, which arrive as an API endpoint and return text.

A paper posted to arXiv on August 3, titled "Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes" (arXiv:2608.02879), takes aim at that gap. Maryam Rezaee, Pooriya Safaei, Maryam Asgarinezhad and Fatemeh Seyyedsalehi describe what they call "a model-agnostic, post-hoc attribution interpreter operating at the sentence level," built to work on proprietary models where the internals are permanently off limits. The unit of explanation is deliberately coarse. Rather than highlighting individual tokens, the method asks which sentences of a prompt carried a given piece of the response, which is closer to the granularity at which people actually write and revise prompts.

Learning the shape of what fits with what

The method leans on an energy-based model, a long-standing idea in machine learning. An energy-based model learns to score how well things go together: compatible pairs get low energy, mismatched pairs get high energy. Train one over enough examples and you end up with a landscape whose valleys mark combinations the data says belong together.

The authors train such a model as a surrogate, one that, in their description, captures "the LLM's internal conceptual consistency between prompts and responses." The point is not to replicate the target model's output, which would be hopeless, but to learn the compatibility structure it exhibits: which kinds of prompt content sit comfortably with which kinds of response content. Attribution then falls out of the landscape. If dropping or altering a sentence moves the prompt and response pair sharply uphill, that sentence was doing real work. If the energy barely shifts, it was not.

Two consequences follow from doing it this way. The first is cost. Conventional black-box attribution methods explain one output by perturbing the input many times and re-querying the model to see what changes, which means every explanation is a fresh round of billed API calls. Here the expense is paid once during training, and the resulting interpreter, which the authors describe as a lightweight standalone tool, then quantifies the influence of prompt sentences on a target output "without additional API calls." The second is scope. Because the surrogate is fit across many examples rather than locally around a single one, the authors argue it "captures broader generation patterns and mitigates instance-specific biases," a known weakness of methods that build a throwaway local approximation for each individual explanation.

Transparency that survives contact with a procurement contract

The practical significance is about who gets to audit what. Regulators, auditors and enterprise risk teams increasingly need to say something defensible about why a deployed model produced a particular output, and in the overwhelming majority of deployments that model is somebody else's, reachable only through an API. Interpretability that requires weights simply cannot be part of that conversation. Methods that work from the outside can be, and making them cheap enough to run continuously rather than as a one-off investigation is what turns them from a research exercise into a monitoring capability.

Sentence-level granularity matters for the same practical reason. A team debugging a prompt that occasionally produces something unwanted does not need a heat map over word pieces. It needs to know which instruction, which retrieved passage, which piece of injected context is responsible, and those arrive as sentences.

An important caveat: the arXiv full text was not retrievable during this reporting, so the evaluation is described here only as far as the abstract states it. No benchmark figures, model names, or baseline comparisons are cited because none could be verified. Everything above rests on the authors' own framing in a preprint that is three days old and has not been peer reviewed.

Explaining the box without the key

The dominant story in interpretability has been about opening models up and reading their internals. This is a bet on the other path: that enough of a model's behavior is recoverable from the outside, by learning the structure of what it treats as fitting together, to give people who will never see the weights a usable account of why they got the answer they did.

Sources

Primary source: "Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes," Maryam Rezaee, Pooriya Safaei, Maryam Asgarinezhad and Fatemeh Seyyedsalehi, arXiv:2608.02879, at https://arxiv.org/abs/2608.02879. Verification notes: the title, author list, abstract, subject category (cs.AI) and submission timestamp (August 3, 2026, 21:01:41 UTC, version 1) were read directly from the arXiv abstract page on two separate retrievals with consistent results, confirming this is a new preprint rather than an older paper resurfacing. The identifier and title pairing was independently corroborated through search results resolving to both the abstract and full-text URLs. The full text at https://arxiv.org/html/2608.02879 could not be retrieved during reporting because of persistent rate limiting, so no quantitative results, evaluated models, datasets or baseline comparisons are reported here. Author affiliations are not listed on the arXiv record and are therefore not stated. All quoted phrases are from the paper's abstract. The work is self-reported and not peer reviewed.