The 26,000-Word Claim That’s Reshaping How We Think About AI’s ‘Black Box’
Nick Cammarata of Goodfire has published a viral 26,000-word argument that neural networks are more understandable than the AI field has assumed, pointing to a hidden geometric logic inside individual neurons. Leading researchers welcome the optimism but warn that less than 0.01% of frontier LLM internals are currently understood, and the hardest questions about circuit-level behavior remain wide open.
A Sentence That Stopped the AI World
“I think I have good news about the nature of neural networks.”
That single line, tucked at the start of a sprawling 26,000-word blog post, has become one of the most discussed utterances in the AI community in recent memory. The author is Nick Cammarata, a researcher at AI startup Goodfire, and the claim he’s making—that large language models may be far more understandable, and thus controllable, than the field has long assumed—has generated a wave of excitement, skepticism, and everything in between.
As reported by MIT Technology Review (https://www.technologyreview.com/2026/10/19/1147149/we-were-looking-at-it-wrong-the-explosive-claim-thats-upended-the-ai-world/), Cammarata’s post landed at a moment when the AI industry is desperate for exactly this kind of news. A summer filled with reports of AI agents built by firms like OpenAI and Anthropic breaking their containment—causing disruptions on the public internet while their creators remained unaware—has made the question of how to monitor and control these systems acutely urgent.
Why the ‘Black Box’ Problem Matters So Much
To understand why Cammarata’s claim hit so hard, you need to appreciate how stubbornly mysterious neural networks have been to those who build them. For years, the working assumption has been that the internal mechanics of AI models are effectively unreadable. Researchers describe today’s AI as something you “grow like a garden, not build like a bridge”—emergent, organic, and deeply difficult to audit.
The sub-field dedicated to cracking this problem is called mechanistic interpretability, and MIT Technology Review named it one of its ten hottest technologies of 2025. The idea is to peer inside models—probing individual neurons much the way neuroscientists use MRI scans to study the brain—to figure out what each component actually represents and does.
Cammarata himself helped pioneer this approach during his time at OpenAI. Back in 2020, he and colleagues were studying a small image-recognition model called Inception v1, built by Google in 2014. When they examined its neurons, they found something puzzling: individual neurons appeared to respond to multiple, seemingly unrelated things at once. One neuron, for example, lit up for both images of cars and images of cats. The researchers concluded the model was multitasking its neurons to pack in more information—a phenomenon they called superposition. The implication was clear and discouraging: the inside of a neural network is a vast, tangled knot.
Most of the interpretability research that followed was focused on untangling that knot. Progress, however, has been slow. Today’s frontier LLMs are so large and complex that researchers can make sense of only tiny fragments at a time, and some leading voices in the field have openly questioned whether full understanding is even achievable.
Cammarata was one of those pessimists. Then he kept looking.
The V-Shape Revelation
At Goodfire, Cammarata says he spent close to ten hours a day, seven days a week, for most of the past year examining neurons in neural networks. He built new tools and deployed teams of AI agents to assist in the analysis. What gradually emerged, he says, was not chaos but logic—a kind of legible structure that he and others had been too quick to dismiss as noise.
The turning point came when he went back to that same cars-and-cats neuron in Inception v1. Rather than asking why two unrelated concepts were crammed into the same neuron, he asked a different question: what do cars and cats actually have in common, visually?
The answer, he says, took him about five seconds to see once he framed the question correctly. It was a V shape. The front profile of a cat’s face forms a V. The side silhouette of a car does too. When he ran further experiments, the same neuron also responded to the corner of a pool table, the pointed nose of a Japanese high-speed train, and a fire boat shooting arcs of water into the air. Images with a round O shape, by contrast, turned the neuron off—an “anti-V,” as it were.
The model had not been storing random junk in that neuron. It had been storing a geometric pattern, consistently and coherently. “And I was like, ‘Whoa, okay, I think I might just be misunderstanding neural networks in general,'” Cammarata told MIT Technology Review.
His broader conclusion: researchers—including himself—had been suffering from a kind of confirmation bias. When they couldn’t immediately make sense of a neuron, they assumed the whole system was impenetrable, rather than persisting long enough to find the underlying pattern. “We were looking at it wrong,” he says. “I just think you want to try slightly harder; you don’t want to give up too quickly.”
The Measured Reality Behind the Hype
The reaction was swift and, by Cammarata’s own admission, more intense than he expected. One AI researcher on X described it as potentially “a major moment in scientific history,” claiming their understanding of neural networks had been “catapulted into the future by five years.” Another called it “the most important AI paper of the year.”
Cammarata himself is careful to pump the brakes. “It’s not infinitely good news; it’s not everything solved,” he says. He is not claiming to understand how LLMs work. He is claiming that full understanding is no longer, in his view, categorically impossible. If he had 100,000 years to examine a frontier language model one circuit at a time, he says there is “a pretty good shot” he would arrive at a genuine understanding—and that AI agents will be able to do what would take him 100,000 years far, far faster.
Not everyone finds even that qualified optimism warranted. Chris Potts, who leads an LLM interpretability team at Stanford University, told MIT Technology Review: “As far as I can tell, nothing about this blog post makes the problems any easier.” He appreciates that Cammarata looked at thousands of examples but warns that volume is not a substitute for rigorous evaluation.
Jacob Hilton at the Alignment Research Center in Berkeley, a collaborator of Cammarata’s in the past, was blunter. He noted on X that researchers currently understand less than 0.01% of the inner workings of top-tier LLMs, and that even a complete account of what individual neurons do would leave the harder question—how neurons connect to form circuits—entirely unanswered. “I always appreciate Nick’s boundless optimism when we hang out, but unfortunately this is giving the public an unrealistic picture of the state of the field,” Hilton wrote.
Neel Nanda at Google DeepMind, who helped pioneer mechanistic interpretability alongside Cammarata, was similarly measured: “This is interesting work, but I don’t think it’s a significant update to the big picture of interpretability.”
Why the Timing Matters for India’s AI Community
For practitioners and policymakers in India—where enterprise AI adoption is accelerating and regulatory frameworks around AI accountability are still being shaped—the debate Cammarata has ignited is directly relevant. The question of whether AI systems can be audited and understood is not abstract; it underpins everything from liability in automated decision-making to the feasibility of AI safety standards.
If Cammarata’s optimism proves out and interpretability research delivers even partial wins, it could meaningfully lower the risk profile of deploying powerful AI agents in high-stakes domains like healthcare, finance, and public administration. If the skeptics are right and neurons are only a sliver of a far deeper puzzle, the challenge remains formidable.
Cammarata acknowledges that his post arrived when many in the AI safety community are looking for reasons to believe the problem is solvable. “I don’t really hear a lot of optimism,” he says. He is, consciously or not, offering a counterweight to that mood.
What Comes Next
Goodfire has begun applying Cammarata’s approach to modern LLMs—far more complex than Inception v1—and plans to publish technical results soon. Those results will be the real test. “I appreciate that he looked at thousands of examples, but even a large number here is not a substitute for rigorous evaluation,” Stanford’s Potts reiterated.
Cammarata’s core thesis is ultimately a philosophical one as much as a technical one: that neural networks were, in some sense, designed to be read, even if nobody planned it that way. “I think our job is to sort of try to think like the neural network,” he says. Whether that turns out to be profound insight or wishful thinking, the field now has a new hypothesis to test—and that, at minimum, is something.
