The Robot That Thought Too Much
Everyone wants to give their company an AI brain. A fifty-year-old argument between two robots suggests they should want the opposite.
July 6, 2026
Two robots, one argument
In 1966, a team at Stanford Research Institute began building the most sophisticated robot the world had ever seen. It was a tower of sensors and circuitry on wheels, connected to a room-sized computer that held something no machine had ever possessed: a complete internal model of its world. Before every move, the robot would consult its model, reason about it, and produce a plan. Life magazine called it “the first electronic person.” The researchers, noting how it trembled when it moved, called it Shakey.
Shakey was a triumph, and Shakey was almost useless. Updating and consulting that beautiful internal model took so long that the robot could spend an hour thinking about a task a child would do without thinking at all. The world would change, the model would lag, the plan would fail. The smarter they made its picture of the world, the further the picture drifted from the world itself.
Twenty years later, a young Australian at MIT named Rodney Brooks committed heresy. He built robots with no model of the world at all. His six-legged machine, Genghis, was a bundle of simple reflex loops layered on top of each other: sense the ground, lift the leg, catch the balance. Nothing in Genghis knew what a room was. Genghis scampered across rough terrain in real time while Shakey’s descendants were still planning. Brooks compressed his heresy into six words that became famous in robotics: the world is its own best model. Don’t build an inner copy of reality and reason about the copy. Check reality itself, constantly, and act.
You can guess which philosophy won. Brooks went on to co-found iRobot. Shakey is in a museum. The Roomba, a machine with barely any brain at all, is in tens of millions of homes.
We are building Shakey again
Now here is the strange part. Half a century later, the most fashionable idea in corporate technology is to build Shakey again. This time for the whole company.
The pitch arrives under many names: the enterprise brain, the organizational memory, the company knowledge graph, the AI that “knows everything about your business.” The idea is always the same. Gather every document, every decision, every fact into one great semantic memory, and let AI consult that memory to get work done. It sounds obviously right. It is the same intuition the Stanford team had in 1966, scaled up ten thousand times.
And the early returns look eerily familiar. Companies have adopted AI almost universally, and the surveys from the big consultancies keep finding the same embarrassing gap: adoption near ninety percent, meaningful financial impact for a minority, most firms reporting little value at all. The models got brilliant. The picture of the company got bigger. The needle barely moved.
The lesson of the Prussian forest
To understand why, it helps to visit an eighteenth-century German forest.
The political scientist James C. Scott tells the story in his book Seeing Like a State. Prussian officials, wanting to manage their forests scientifically, replaced the chaotic old woodland with something legible: a single species of tree, planted in perfect rows, every trunk countable, the whole forest reduced to a table of numbers. For one generation it was a spectacular success. Yields were predictable, administration was effortless, the map matched the territory because the territory had been forced to match the map. Then the soil, deprived of the messy underbrush and dead wood the old forest had lived on, gave out. The second and third plantings sickened. German foresters had to invent a new word for what happened next: Waldsterben. Forest death.
Corporate life has run Scott’s experiment roughly once a decade for forty years. Expert systems in the eighties. Knowledge management in the nineties. Corporate wikis, enterprise ontologies, master data programs. Each began with the same promise: at last, one legible map of everything we know. Each ended the same way. The people assigned to tend the map stopped tending it, because tending it was a chore and no one’s actual work depended on it. The map drifted from the territory. Eventually a wrong map became worse than no map, and the whole thing was quietly abandoned. Anyone who has clicked into a company wiki and found the org chart from three reorganizations ago has stood in a dead Prussian forest.
Two kinds of memory, and only one survives
There is a simple way to see the underlying law. In most households there are two kinds of memory. There is the calendar on the fridge, and there is the journal someone swore they would keep. The calendar is accurate to the day, because everyone uses it, and if an entry is missed a child misses a dentist appointment. The journal is three months behind, because keeping it is a virtue rather than a necessity. Memory kept true by use survives. Memory kept true by effort decays.
Companies are full of the first kind, and this is the fact the enterprise-brain pitch skips over. The accounting ledger is accurate because money actually moves through it. The order system is accurate because shipments actually route through it. Payroll is accurate because people get very loud when it isn’t. A company, it turns out, has already written itself down, in the systems where the work actually happens, and that record stays fresh for the same reason the fridge calendar does. Nobody maintains it. The work maintains it.
Which raises an awkward question about the giant AI memory: what exactly is it for? It is a photocopy of the filing cabinet, pasted into a diary, and the diary starts aging the moment the copying stops.
The cashier does not need the CEO’s strategy
There is a second thing the brain-builders skip, and it may be the more important one. To do almost any job in a company, you do not need to know everything about the company. The cashier makes change without reading the marketing plan. The claims adjuster settles the claim without the CEO’s calendar. This is not a bug in corporate life. It is the entire technology of corporate life. Over a century, firms painstakingly arranged themselves so that every decision needs only a small slice of information: this form, these rules, that record. Herbert Simon won a Nobel prize for explaining why this works. Friedrich Hayek won one for showing that whole economies coordinate this way, through small local signals, with no central mind anywhere. Ronald Coase won one for demonstrating that reducing how much any one person needs to know is, in a sense, the reason firms exist at all. Three Nobel prizes converging on one lesson: large-scale coordination runs on structure and local knowledge, not on a big brain. The organization that wires every action to a memory of everything is not upgrading itself. It is un-inventing itself.
What to build instead
So what does the alternative look like? Roughly what Brooks built, and roughly what a good restaurant kitchen has always been. Not one chef who knows everything, but stations. Small loops, each responsible for one concrete result. Each loop can suggest an action but cannot simply take it: the action is checked against written rules that work like a guest list, where nothing happens unless a rule explicitly allows it. Each loop confirms its work against reality itself, the actual ledger, the actual order system, rather than against its own account of what it did. Each loop keeps receipts no one can quietly edit. And each loop earns independence the way a new employee does, by watching first, assisting under supervision, and acting alone only after a track record, with every human correction becoming a permanent rule so the same question never needs answering twice. The system that results holds more memory than any brain, but every bit of it is fridge-calendar memory, written by the work itself.
The builders are already converging
The people building AI’s infrastructure appear to be arriving at the same place. Michael I. Jordan of Berkeley, once identified by the journal Science as the world’s most influential computer scientist, has spent years arguing that the future is not one artificial mind but intelligent infrastructure, systems that are smart the way markets are smart. A market that feeds New York every single day, he likes to point out, is an intelligent entity, and no neuron in it knows the whole plan. NVIDIA’s researchers published a position paper arguing that fleets of small, specialized models will beat giant ones for real work, describing the winning design as Lego-like composition. Andrej Karpathy, formerly of Tesla and OpenAI, tells builders that the products which actually work keep humans in tight verification loops and expand a machine’s autonomy the way you’d expand a trainee’s. Different vocabularies, same instinct. Shakey lost. Genghis won. Build loops, not brains.
The brain you already have
None of this means the dream of organizational memory is worthless. It means the executives asking for a brain already have a better one than the vendors are selling. It is sitting in the transaction systems that never go stale, in the audit trails, in the rulebooks that encode every hard lesson the institution ever learned. A company was never a person, and it never needed a mind. It is something rarer: an institution, a set of roles, rules, and records that lets thousands of people, and soon thousands of machines, accomplish what no single mind could hold.
The Prussians wanted a forest they could read. They should have wanted a forest that could live. Fifty years of robots, forty years of dead corporate wikis, and three Nobel prizes are all whispering the same advice to anyone about to sign the purchase order for an enterprise brain.
The world is its own best model. Your company is too.
Research Notes
The evidence behind “The Robot That Thought Too Much”: what the surveys, benchmarks, vendor documentation, and analyst record from 2022 through mid 2026 actually show. Sources at the end. The most fragile claims are flagged rather than hidden.
Note 1. The value gap is measured, not anecdotal
AI adoption is nearly universal while financial impact remains rare, and the numbers come from organizations with no stake in the argument.
McKinsey’s State of AI research finds roughly 88 percent of organizations now use AI in at least one function, while only about 39 percent report any earnings impact at all, most of it small. Deloitte’s State of AI 2026 survey of 3,235 executives finds only 34 percent of organizations describe themselves as deeply transforming with AI, and only 21 percent have a mature governance model for AI agents despite the vast majority planning to deploy them. BCG’s global research finds a majority of companies reporting little to no value from AI investment, with a single-digit minority capturing value at scale. Gartner, in a widely reported June 2025 release, projected that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.
One widely quoted figure deserves a flag. The MIT NANDA study’s finding that 95 percent of enterprise GenAI pilots produced no measurable P&L impact has had its methodology publicly challenged for a narrow success definition and a short measurement window. It is directionally consistent with the sturdier McKinsey, BCG, and Deloitte numbers, but it should be quoted with attribution rather than asserted as fact.
Note 2. Bolting on fails and redesign works, according to the people selling both
AI added to work designed for humans produces little; work redesigned around AI produces much. This is now the explicit, published position of the major consultancies and platform vendors themselves.
McKinsey’s agentic AI research states that value comes from reimagining workflows from the ground up with agents at the core, not from deploying tools into existing processes. Gartner’s guidance uses nearly identical language about rethinking workflows from the ground up. Microsoft’s Work Trend Index research has pivoted its framing from individual copilot productivity to the “Frontier Firm,” an organization redesigned around orchestrated agent operations under human direction. The most concrete number comes from BCG’s June 2026 report on reinventing the operating system of work: in its client deployments, end-to-end process redesign achieved cost reductions above 60 percent where bolt-on tool deployment did not, and almost two thirds of the agentic AI leaders it surveyed expect operating model changes as a consequence. These are consulting-house figures with selection bias toward successful engagements, and they should be read as directional. The direction, however, is unanimous.
Note 3. The benchmark record argues for verified loops, not free-running autonomy
Checking work against reality, rather than trusting an AI’s own account, is the load-bearing design principle, and the agent-benchmark literature makes the case quantitatively.
On tau-bench, a benchmark of realistic customer service tasks involving tools, policies, and databases, state-of-the-art models succeed on fewer than half of tasks, and consistency collapses when the same task is attempted repeatedly: pass rates over eight attempts fall below 25 percent in the retail setting, a drop of roughly 60 percent from single-attempt scores. Tasks requiring four or more database writes succeed only around one fifth of the time. On WorkArena, an enterprise UI benchmark built by ServiceNow Research, GPT-4o completed about 43 percent of tasks and scored zero on an entire task category. On the original WebArena benchmark, the best GPT-4 based agent achieved 14.4 percent end-to-end success against 78.2 percent for humans. Agent-SafetyBench reported that none of the tested agents exceeded a 60 percent safety score. AgentBench identified long-horizon reasoning and instruction following as the persistent obstacles.
The pattern across five independent benchmarks: failures concentrate exactly where trust should not be extended, in multi-step write operations against real systems, in policy adherence, and in long chains of unverified action. The same literature shows what helps. The foundational ReAct paper demonstrated that interleaving reasoning with action and observation improves reliability, and Reflexion showed that structured feedback loops improve performance further. Loops with checking beat straight-line autonomy in the lab as well as in the argument.
Note 4. The industry’s own architecture converged on the loop
The strongest evidence is architectural rather than statistical. Inspected through developer documentation and SDKs rather than marketing, every major vendor’s production agent stack now exposes the same set of primitives: a trigger, scoped context, a reasoning or proposal step, a policy or guardrail gate before action, human approval and escalation points, write-back to systems of record, execution traces, evaluation harnesses, and a learning loop. This holds across OpenAI’s AgentKit, Anthropic’s agent guidance and the Model Context Protocol, Microsoft’s multi-agent orchestration, Google’s agent development kit with its explicit loop agent, Salesforce’s Agentforce with its observability layer, ServiceNow’s orchestrator and control tower, AWS Bedrock AgentCore’s policy enforcement, IBM’s watsonx Orchestrate, and the open-source runtimes LangGraph and AutoGen.
Anthropic’s published engineering guidance is representative of the design philosophy: the best production systems begin as simple, composable workflows, adopting fuller autonomy only where task variability justifies the added cost and risk. The Berkeley “compound AI systems” thesis, among the most cited artifacts in the field, argues that state-of-the-art results now come from systems of components rather than from single monolithic models. NVIDIA’s research position paper argues that small, specialized models are sufficient, more suitable, and more economical for most agentic invocations, describing the winning pattern as composition of small experts rather than scaling up a monolith. None of these organizations uses the kitchen metaphor. All of them are describing the kitchen.
Note 5. Even the brain vendors gate the writes
The claim that a global company memory should never hold authority over action has a revealing witness: the leading vendor of the organizational-brain archetype itself. Palantir’s ontology is the most serious and well-funded version of the enterprise semantic layer in existence, and in Palantir’s own platform documentation, “all writes must go through Action Types,” which enforce validation rules, approvals, and audit trails, with human-in-the-loop workflows woven through autonomous operations. The semantic layer informs; a governed action layer decides. Across the major vendors, none positions a global semantic memory or knowledge graph as the authoritative execution layer for operational decisions. Knowledge graphs and vector stores appear consistently as context providers on the read path. Where the map exists, the map advises. The gate still decides.
Note 6. The honest caution: naming things is not building them
The counter-evidence belongs in any honest appendix. Gartner has coined the term “agent washing” for the rebranding of chatbots, assistants, and RPA as agents, estimating that of the thousands of vendors claiming agentic products, only around 130 are genuine. The market narrative runs well ahead of operational reality: most enterprises remain at assistant-level maturity, copilots dominate actual adoption, and the gap between claimed and production-scale agent deployment is the defining feature of the current market. The governed-loop architecture is where the evidence points, not where most organizations stand. Anyone presenting it as an already-won category, rather than an emerging direction with a measured rationale, is committing the same inflation the data punishes.
Note 7. The theory predates the technology
The intellectual claims are standard results in their home fields, documented there long before the AI literature arrived: Herbert Simon on near-decomposability and bounded rationality, Friedrich Hayek on coordination through local signals, Ronald Coase and Oliver Williamson on the firm as an information-economizing institution, W. Ross Ashby on regulators scoped to what they regulate, Rodney Brooks on behavior-based robotics and the world as its own best model, James C. Scott on the failure of centrally legible maps, and Michael I. Jordan on intelligence as a property of markets and loosely coupled systems. A 2020s engineering literature converging on conclusions reached independently by cybernetics, economics, robotics, and political science across seventy years is itself a finding.
Sources
Yao et al., ReAct (ICLR 2023). Shinn et al., Reflexion (2023). Yao et al., tau-bench (arXiv 2406.12045, 2024). WorkArena, ServiceNow Research (ICLR 2024). Zhou et al., WebArena (2023). AgentBench (ICLR 2024). Agent-SafetyBench (2024). Zaharia et al., The Shift from Models to Compound AI Systems, Berkeley AI Research (2024). Belcak et al., Small Language Models are the Future of Agentic AI, NVIDIA Research (arXiv 2506.02153, 2025). Anthropic, Building Effective Agents (December 2024). Gartner press release on agentic AI project cancellations and agent washing, as reported by Reuters (June 2025). McKinsey, The State of AI (2025). BCG, Reinventing the Operating System of Work with AI (June 2026). Deloitte, State of AI 2026 (survey of 3,235 executives). Microsoft, Work Trend Index, Frontier Firm research (2025 to 2026). MIT NANDA, The GenAI Divide (2025; methodology contested, quote with attribution only). Palantir platform documentation on Action Types. Michael I. Jordan, Artificial Intelligence: The Revolution Hasn’t Happened Yet, Harvard Data Science Review (2019). Simon, The Architecture of Complexity (1962). Hayek, The Use of Knowledge in Society (1945). Brooks, Intelligence Without Representation (1991). Scott, Seeing Like a State (1998).