The Cost of AI Is Collapsing. Can Your Organisation Respond?

Inference costs are falling tenfold every year. Yet 56% of CEOs report no measurable benefit from AI. The bottleneck has moved from the AI model to the operating model.

At three o’clock on the afternoon of 4 September 1882, Thomas Edison stood in J.P. Morgan’s office on Wall Street and gave the signal. Half a mile away at 257 Pearl Street, his chief electrician closed the switch on six 27-ton dynamos. Four hundred incandescent lamps lit up across Lower Manhattan, including Morgan’s own desks and the offices of the New York Times. The world’s first commercial central power station was online, and the cost of industrial energy had begun a forty-year collapse.

For the first thirty of those forty years, almost nothing happened to productivity. By 1900, electric motors drove fewer than 5% of American factory mechanical power. Output per worker in the US and UK was slowing. A contemporary observer might fairly have remarked that the dynamos were everywhere but in the productivity statistics.

Sound familiar?

In November 2022, a system at GPT-3.5’s level cost roughly $20 per million tokens. By October 2024, the same capability cost less than $0.07 — a 280-fold collapse in 24 months that Stanford’s AI Index calls the steepest decline of any input cost in the history of enterprise computing. Andreessen Horowitz calls it LLMflation: a tenfold annual decline. Epoch AI finds a median 200x annual decline since January 2024, and Anthropic now reports approximately 4% of all GitHub commits worldwide are authored directly by Claude Code.

If you are a board director who has been tracking AI primarily through your IT budget line, this is the news you came for: AI is rapidly becoming free. The strategic question is no longer whether you can afford AI. It is whether you can use it.

The Puzzle

PwC’s 29th Global CEO Survey, polling 4,454 chief executives in early 2026, found that 56% have realised no revenue or cost benefit from AI; only 12% reported gains on both. That number has not meaningfully moved in two years. In any other market, a 280-fold collapse in input costs would have produced a productivity tsunami. So where, exactly, has all the AI value gone?

The most common defence — and one I am willing to partly concede — is the productivity J-curve. Erik Brynjolfsson is right that technologies requiring deep organisational change show a decade or more of investment before returns appear. The canonical example is the one we just opened with. In his 1990 paper The Dynamo and the Computer, Stanford economic historian Paul David identified what finally broke the 1900 puzzle. Factories had to abandon “group drive” — replacing the steam engine with one big motor turning the same overhead line shaft — and adopt “unit drive”, with a small motor on every machine. That sounds trivial. It wasn’t. Unit drive meant flipping the building from multi-storey-around-a-shaft to single-storey-around-the-flow-of-work: new floor plans, new training, new contracts, new managerial logic. It took thirty years and the labour shock of the First World War to push American factories through that redesign. When they finally made the switch, manufacturing productivity rose at roughly 5% a year through the 1920s.

The argument is partially correct. The AI returns will arrive; they may simply be slow. But it misses a more uncomfortable finding which surfaced in 2026 data.

Same Input, Opposite Output

The most important data point in enterprise AI right now is variance, not aggregate productivity. A recent DORA-style study covering 4.2 million developers across 67,000 organisations between November 2025 and February 2026 found the same AI tools producing opposite outcomes depending on the organisation: in well-structured firms, AI was a force multiplier — accelerating delivery, halving customer-facing incidents; in struggling firms, it roughly doubled incident rates.

This is not what a J-curve would predict. A J-curve says everyone catches up. The data shows a widening bifurcation — exactly the 1900–1925 pattern: same dynamo, opposite outcomes, depending entirely on whether the firm redesigned around the new technology or merely bolted it onto the old. BCG’s 2025 Build for the Future study finds the 5% it classifies as ‘future-built’ extracting 2.7 times the AI ROI of everyone else; the gap is widening, not closing.

Even METR’s headline sceptical study now points the same way. Their July 2025 trial famously found experienced developers 19% slower with AI tools; their February 2026 update finds the same cohort 18% faster, and describes this as a lower bound — 30–50% of high-uplift developers declined to participate because they no longer wanted to work without AI.

The reason most enterprises see no AI returns is not that AI is not good enough. AI is more than good enough. Most enterprises are not yet good enough at being changed by AI.

Operating Model Debt

The phenomenon needs a name. I will call it Operating Model Debt (OMD) — the accumulated cost of running a 21st-century intelligence layer on a decision-rights architecture designed for the 2010s. The 1900 line-shaft factory is the cleanest image: every machine clamped to the wrong axis is an interest payment on the old architecture, and the bill cannot be settled one machine at a time.

OMD compounds like technical debt: invisible on any single project, decisive over a portfolio, paid in the currency that matters most to AI — speed of organisational response. It is the deeper structure under what I described in Counting Cards as the AI Velocity Paradox — code shipped faster than the business can safely ingest it.

The signature is everywhere. Insight Enterprises recently disclosed that one of its own AI agents was built using AI in three weeks — and then required three months of change-management vetting to deploy. That four-to-one change-to-build ratio is becoming normal even at sophisticated AI-native firms. Celonis, surveying 1,649 firms in February 2026, ranked the top three blockers to AI in production: a 47% expertise gap, 45% departmental misalignment, 45% inability to translate business context for AI. None was technical.

The Walmart Parable

The dynamo is not the only parallel. The closest analogue in living memory is also an operating model story.

In 1998, Walmart had every advantage in e-commerce: $100 billion of revenue, the best supply chain on earth, and a brand most Americans trusted more than their bank. Amazon was a books startup with no profits. That year, a Walmart executive named Robert Davis walked into CEO David Glass’s office to argue e-commerce should be a strategic priority. He was ignored. Shortly afterwards, Jeff Bezos poached Walmart’s IT chief, Rick Dalzell, to become Amazon’s first CIO.

For eighteen years, Walmart treated e-commerce as a feature of its existing operating model. Each initiative had to be negotiated through buyers, regional managers and supply-chain functions whose decision rights remained intact. Most stalled. By 2016, Walmart paid $3.3 billion for Jet.com and installed its founder Marc Lore as CEO of Walmart eCommerce. Walmart had better technology than Jet. The decisive move was importing an operating model because the existing one would not bend.

Walmart did not lose to Amazon for a quarter-century because its stores were bad. It lost because its operating model was optimised for a world that ended in 1998 — and the OMD compounded faster than it could pay down.

The Agile Inversion

The dynamo and Walmart stories are the playbook running in your firm right now. There is a specific reason most boards are unprepared for it.

The agile revolution of the 2010s — Spotify squads, two-pizza teams, data mesh, DevOps autonomy — federated decision rights aggressively, because software was the bottleneck and local speed beat global coherence. AI inverts the maths. AI value depends on what crosses every team boundary: integrated data, enterprise-wide model risk, workflow redesigns that span functions, regulatory exposure that lands at the board, a talent reshape that touches every P&L. Federated decision rights — your great asset in the software era — have become your largest hidden contributor to OMD. The 5% are not winning because they are more agile. They are winning because they figured out which decisions to re-centralise.

Rails Common, Trains Local

The risk of any ‘re-centralise’ argument is that it sounds like 1980s command-and-control. It is not. The principle is more subtle:

For every AI-critical domain, centralise the architecture; federate the content. Centralise what must be consistent for AI to work. Federate what must remain local for AI to deliver value in context.

Take data. Each business unit must own its inputs and last-mile interpretation, because nobody else is close enough to spot garbage going in. But the infrastructure — lakehouse, lineage, access policy — must be common. So must the governance: KPIs, definitions, reporting standards. So must the data model: what we mean by ‘customer’, ‘revenue’, ‘active user’. The rails are common. The trains are local.

The same pattern applies elsewhere: model risk (framework central, the model-by-model judgement local); workflow authority (methodology central, what to redesign local); talent (role architecture central, hiring local).

Most organisations have this exactly backwards. They federated the rails - each business unit built its own data model, hired its own translators, ran its own model-risk approach - and tried to centralise the trains, typically through a Centre of Excellence hand-building use cases for every business unit. The COE produces decks. The line refuses to absorb the cost of change. The OMD compounds.

The Four Questions

The board’s job in 2026 is to govern the gap between collapsing AI cost and rising OMD. That governance reduces to four questions. A board that cannot answer them plainly is not governing AI. It is hoping.

• Data. Do we have a single, governed data model — with one definition of ‘customer’, ‘revenue’, ‘active user’ — and a single accountable owner for data infrastructure, lineage and quality across the firm?

• Model risk. Is there a single executive owner of our model inventory, risk thresholds and audit trail, with an enterprise view of cumulative exposure? Or is model risk being assessed locally, by the same teams shipping the models?

• Workflow authority. Who has the authority to redesign an end-to-end workflow that crosses three functions, retire two roles, and redraw the org chart inside a single quarter? If the answer involves unanimity from function heads with veto rights, you do not have an AI operating model. You have a coalition.

• Talent and operating model. Is there an enterprise architecture for AI-era roles — what they are, how careers move through them, how reskilling is delivered — owned at the centre? Or is every business unit writing its own AI-PM job description and competing for the same scarce translators?

Throwing the Switch

Electrification took roughly forty years to ripple through the productivity statistics. E-commerce took twenty-five. This is a real lag, not an excuse — the board that responds to AI in 2027 will probably still survive. But both stories carry a darker lesson: the factories that moved early to unit drive pulled away and stayed away. Walmart could afford to be ten years late to e-commerce because Amazon’s logistics was not yet a moat. By 2010, it was. The window of late-but-survivable closes silently, and only in retrospect.

The cost of code is going to zero. The cost of inference is going to zero. The cost of intelligence itself is, on a long enough horizon, approaching zero. None of that will save the firms whose Operating Model Debt compounds faster than they can pay it down.

In 1900, an observer might fairly have said the dynamos were everywhere but in the productivity statistics. The puzzle was real — and was solved, but only by the firms willing to redesign the factory rather than bolt new technology onto the old. The question on your next board agenda is no longer ‘what will AI cost us?’ The cost answer is, more or less, nothing. The question is whether your organisation can respond.

Your model is no longer the bottleneck. Your operating model is.

Footnotes & Sources

• IEEE / Engineering and Technology History Wiki, Milestones: Pearl Street Station, 1882. World’s first commercial central power station, opened 4 September 1882 at 257 Pearl Street, Manhattan; six 27-ton “Jumbo” dynamos serving an initial 400 lamps at ~85 customers including J.P. Morgan and the New York Times.

• Paul A. David, The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox, American Economic Review 80:2, May 1990. Source for the productivity-paradox analogy, group-drive vs unit-drive, the role of WWI as catalyst, and the claim that fewer than 5% of US factory mechanical drive was electric in 1900.

• Stanford HAI, 2025 AI Index Report. Inference cost for GPT-3.5-level performance dropped 280-fold between November 2022 and October 2024.

• Andreessen Horowitz, Welcome to LLMflation, 2024. Approximately 10x annual cost decline for equivalent LLM performance.

• Epoch AI, LLM inference prices have fallen rapidly but unequally across tasks, March 2025. 200x median annual decline since January 2024.

• Anthropic, Claude Code commit data, 2026. Approximately 4% of GitHub commits worldwide authored by Claude Code.

• PwC, 29th Global CEO Survey, January 2026 (n=4,454). 56% of CEOs report no revenue or cost benefit from AI; only 12% report both.

• DORA / DX Research (cited via Tacho, 2026), telemetry analysis of 4.2 million developers across 67,000 organisations, November 2025–February 2026. Same AI usage producing opposite incident-rate outcomes — up to 50% reduction in well-structured firms, up to 2x increase in struggling firms.

• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025 (initial 19% slowdown finding); updated study, February 2026 (18% speedup for same cohort, described as a lower bound due to selection effects).

• BCG, Build for the Future 2025 Global Study (n=1,250 firms). 5% of firms classified as ‘future-built’ achieve 2.7x the AI ROI of others; gap widening over the last 12 months.

• Insight Enterprises, cited in TechTarget IT Operations, March 2026. AI-built MVP completed in three weeks; change-management vetting took three months — a 4:1 change-to-build cost ratio.

• Celonis, Process Optimization Report, February 2026 (n=1,649 firms). Top three blockers to AI in production: 47% expertise gap, 45% departmental misalignment, 45% AI struggling to understand business context.

• Brynjolfsson, Rock & Syverson, The Productivity J-Curve: How Intangibles Complement General Purpose Technologies, NBER Working Paper, 2018.

• Walmart corporate filings, acquisition of Jet.com for $3.3 billion, August 2016; Marc Lore tenure as CEO of Walmart US eCommerce, September 2016 – January 2021.

• Jason Del Rey, Winner Sells All: Amazon, Walmart, and the Battle for Our Wallets, HarperCollins, 2024 (Robert Davis / David Glass episode of 1998; Bezos’s recruitment of Rick Dalzell).