“We were looking at it wrong”: The explosive claim that’s upended the AI world

Nick Cammarata, a researcher at the interpretability startup Goodfire and a former OpenAI researcher who helped pioneer the field, published a 26,000-word blog post arguing that large language models may be far easier to understand, and therefore to monitor and control, than most experts have believed. In an interview with MIT Technology Review, he said he spent most of the past year looking at individual neurons, often with teams of AI agents helping.

If he's right, it would chip away at one of the field's bleakest assumptions: that the insides of these models are a permanently tangled knot. That matters right now because the industry has spent the summer dealing with agents from major labs slipping their containment and causing trouble online, often before the companies noticed.

We find the core story genuinely compelling. Back in 2020, Cammarata and colleagues found a neuron in Google's Inception v1 image model that seemed to respond to both cars and cats. That kind of finding helped establish superposition, the idea that models cram unrelated concepts into single neurons. Revisiting it, he now argues the neuron was detecting a V shape, the front of a car in profile and the face of a cat, and that it switched off for round O shapes. Same images, different assumption, and suddenly the neuron makes sense. That's a lovely example of how expectations shape what scientists see.

But this is one vivid example on a 2014 model, plus a researcher's informed hunch. The modern-LLM results haven't been published yet. Stanford's Chris Potts says nothing in the post makes the hard problems easier. The Alignment Research Center's Jacob Hilton notes researchers understand less than 0.01% of what happens inside top models, and that real understanding depends on how neurons combine into circuits, not just on neurons alone. Google DeepMind's Neel Nanda calls it interesting but not a significant update to the big picture. Cammarata himself says people have taken it too far.

So why the frenzy? Cammarata offers the honest answer: the safety community is starved for good news. That's understandable, and it's also exactly when a field should be most careful. Hope is not a measurement. If labs, investors or regulators start treating interpretability as nearly solved, they could ease off on the other safeguards that matter while it isn't.

There's a real upside worth naming. If even part of this holds, interpretability could become a practical tool for diagnosing why a specific model misbehaves, which Nanda agrees is plausible. And Cammarata's bet that automated AI scientists could compress centuries of circuit-by-circuit work into something tractable is the kind of pro-useful-AI argument we'd love to see proven. Using AI to make AI legible is a good use of the technology.

Our read: this is an encouraging reframing, not a breakthrough yet. I'm watching for the technical write-up on LLMs and for independent groups trying to replicate it. Until then, the safest assumption is the old one: we don't understand these systems well enough, and we should act like it.

Mechanistic interpretability tries to reverse-engineer what individual parts of a neural network compute, a bit like neuroscientists probing brain activity. Superposition, the idea that networks pack multiple unrelated concepts into single neurons, became a foundational assumption after early work on models like Inception v1. Cammarata and Goodfire say they plan to publish technical results on modern LLMs soon.