Why AI Pilots Fail: What 15 Studies Tell Us About the Pilot-to-Production Gap

In August 2025, MIT's NANDA initiative released a study that hit the business press harder than any AI research since the ChatGPT launch. The headline: 95% of generative AI pilots at companies yield no measurable P&L impact. Thirty to forty billion dollars of enterprise investment, and the vast majority produced no return that showed up in the financials.

Fortune, the Wall Street Journal, CIO, and every LinkedIn thought leader spent the following month trying to explain it. Some blamed hype. Some blamed vendors. Some argued the models simply weren't good enough yet, and another generation would fix it.

The more interesting read is that MIT is not an outlier. It is the clearest data point in a much larger pattern.

Across the last eighteen months, BCG, McKinsey, Gartner, RAND Corporation, HBR, Deloitte, KPMG, EY, Stanford, and Zapier have all run their own studies with different samples, different methods, and different definitions of success. They converge. Somewhere between 74% and 95% of AI pilots never reach production at scale. And when the researchers dig into why, they find the same three problems, in the same rough proportions, every time.

This piece walks through those findings, what the 5% of companies doing it well have in common, and what all of it means for a creative business trying to bring AI into its operations without burning a year and half a budget to find out nothing works.

Chapter 1: The Pilot-to-Production Gap

Set the MIT number aside for a moment and line up the rest.

BCG surveyed 1,250 CxOs across 68 countries for its September 2025 Widening AI Value Gap report. Only 5% of companies were capturing AI value at scale. Sixty percent reported hardly any measurable value at all, despite real spend.

McKinsey's 2025 State of AI found that 78% of organizations are now using AI in some way, up from 55% the year before. But only 39% reported any enterprise-level EBIT impact, and most of those said less than 5% of their EBIT could be attributed to AI. Just 6% qualified as high performers, meaning companies where AI moved the bottom line 5% or more.

Gartner, working from a July 2024 survey of 822 business leaders, predicted that 30% of generative AI projects would be abandoned after proof of concept by the end of 2025. The reasons cited: poor data quality, inadequate risk controls, escalating costs, and unclear business value. Rita Sallam, the Gartner analyst behind the prediction, put it plainly: executives are impatient to see returns, and organizations are struggling to prove and realize value.

RAND Corporation ran a narrower but deeper study. They interviewed 65 experienced data scientists and engineers who had spent five or more years building AI systems in industry and academia. Their finding: more than 80% of AI projects fail, twice the failure rate of non-AI IT projects. The root causes were not technical sophistication but organizational basics, which we'll come to.

Zapier's 2025 Enterprise AI Report, based on 550 C-suite executives at companies of 1,000 or more employees, found that 74% of AI pilots stall before scaling and 70% of enterprises fail to integrate AI tools beyond basic connections.

HBR's November 2025 piece, Most AI Initiatives Fail, cited research showing 87% of AI projects never get deployed in real-world production. The authors named the dynamic directly: organizations deploy AI solutions department by department without linking them to enterprise goals, producing technically successful implementations that never reach production.

That list is not cherry-picked. It is the consensus view of the last year and a half of primary research from the most credible institutions publishing on AI in business. The specific percentages vary because the populations vary. The pattern does not.

No one owns it. No one can feed it the right data. No one can tell whether it is better or worse than what the team was doing before. And nobody has the time to find out.

The question is not whether the gap exists. The question is what is actually in it.

Chapter 2: Context Is the Whole Game

When RAND asked its 65 practitioners what actually kills AI projects, the top two answers were not technical. They were about context.

The first: stakeholders misunderstand or miscommunicate the problem the AI is supposed to solve. Teams optimize models for the wrong metric, or for a problem that looks similar to the real one but isn't. The second: organizations lack the data to train, fine-tune, or ground the models against their own business reality.

This is the same finding MIT reached through a very different door. NANDA's researchers studied 300 public AI deployments and found that AI purchased from specialized vendors and built as a partnership succeeded about 67% of the time. Internal builds succeeded about a third as often. The difference was not that vendors had better engineers. It was that vendors arrived with a template for plugging the model into a business context, and internal teams often had to invent that layer from scratch and usually didn't finish.

HBR's October 2025 piece Why Agentic AI Projects Fail quantified the modern version of this problem. 42% of enterprises need access to eight or more separate data sources to deploy an agent successfully. 79% expect data challenges to impact their rollouts. Only 15% of leaders say their data and systems are fully ready to support agentic AI. Only 20% say their technology infrastructure is ready. Only 12% feel risk and governance controls are in place.

KPMG's Q3 2025 Pulse Survey reinforced it from the top: 82% of leaders named data quality as the critical barrier to AI ROI. That is not a research finding that flatters any tool vendor. It is a finding about the state of the private data inside the business.

Here is the thing about context. The large foundation models were trained on the public internet. They know a great deal about the world in general and nothing at all about your world specifically. They do not know your clients, your pricing, your last year of email, your playbook, your standard scope, your usual vendors, your designer's preferred suppliers, the way your partner talks to a panicked homeowner three days before a shoot. That private context is where the value lives, and most pilots never connect to it.

Two patterns show up when you look at the small set of pilots that cross into production.

The first is that someone did the unglamorous work of making a single source of truth before anyone touched a model. They consolidated client records out of three half-used CRMs. They moved project notes out of nine places into one. They wrote down the scope template everyone was pretending they had. The pilot came second.

The second is that the AI was asked to do a job narrow enough that the context it needed was knowable. Not "improve our marketing," which requires the AI to know everything about the company. But "take the transcript of this discovery call and fill the intake form that leads to the proposal draft." A job where the inputs, outputs, and quality check are all defined.

Both of these are context moves. The AI itself is the easiest part of the stack. The hardest part is giving the AI the same situational awareness a competent new hire would have after their first month, in a form the AI can actually read.

Chapter 3: The Governance Vacuum and the Shadow AI Problem

While leadership debates policy, the team does not wait.

Across the 2025 research, somewhere between 50% and 71% of employees are already using AI tools that were never approved by their IT or operations function. SecurityWeek and CIO both reported that enterprise leaders, the ones ostensibly making the policy, are among the heaviest shadow AI users. A widely cited figure is that 63% of employees believe it is acceptable to use AI without IT approval. Zapier found that 31% of enterprises discover new rogue AI tools in their organization every month.

Deloitte's State of Generative AI in the Enterprise study found that 69% of companies expect implementing a full governance strategy to take more than a year. That is not a posture anyone chooses. It is what happens when AI adoption moves faster than the committees that exist to oversee it. The same study found regulatory compliance concern rising from 28% in Wave 1 to 38% in Wave 4 as companies started to realize how exposed they were.

EY's 2025 AI Pulse found 87% of senior leaders reporting barriers to agentic AI adoption, led by cybersecurity, data privacy, and policy gaps. KPMG's Q3 2025 data showed 65% of leaders naming scaling use cases as their top ROI barrier, with 62% naming workforce skills and 78% naming cybersecurity.

The compounding effect is the real issue. A sanctioned AI pilot can starve for context while unsanctioned AI use leaks the same private context through the side door. Client contract text pasted into a free chatbot. Financials pasted into a summarizer. The pilot that was supposed to be the careful first step has become the slowest, least trusted version of what is already happening in the business.

Governance in this context is not about writing a twelve-page policy that nobody reads. It is about answering four operational questions with enough clarity that the team can act without guessing:

1. Which categories of work is AI allowed to touch, and on which systems.
2. Who owns the evaluation of whether a given AI output is good enough.
3. Where the data the AI uses comes from, and whether we are allowed to feed it there.
4. What happens when the AI is wrong.

Those four questions are uncomfortable because they force a business to admit how much of its operating model was implicit. That is part of why governance keeps getting deferred. The pilot is exciting. The policy is uncomfortable. So the pilot ships first and the policy never quite ships.

Most of the failed pilots in the data sets above died in exactly this hand-off. A fifth of them never should have been pilots in the first place, because the governance questions made it clear the company was not yet allowed to do the thing. Another third got built, worked in isolation, and stalled at production because nobody had resolved who would own the thing in steady state.

The Turn: Pilots Aren't the Problem. Pilots Without Runway Are.

Return to the original 95% number with everything we now know next to it.

The pilots that failed were not, on the whole, technical failures. Most of them worked in the narrow sense that matters to a data scientist: the model returned a reasonable output on the test set. They failed because they had no runway between "the model works" and "the business works." The runway is the thing between them. The runway is operations.

Three design principles keep showing up in the 5% of companies that are making AI pay off, stated differently in every study but pointing at the same underlying shape.

1. Audit before you automate

McKinsey found that the single strongest predictor of EBIT impact from AI is fundamental workflow redesign, not tool selection. 55% of AI high performers fundamentally redesigned workflows when deploying AI. Only about 20% of other firms did. BCG found that 70% of realized AI value comes from core business functions like sales, delivery, and production, not from support functions like HR or IT where most pilots tend to live. You cannot redesign a workflow you have not understood. An operations audit, done honestly, is the cheapest thing an AI program can start with, because it tells you which workflow is actually ready for a model and which one needs to be cleaned up first.

2. Context before compute

The foundation models are a commodity. The context you feed them is the asset. That means the ground truth of your business, the client records, the scope templates, the pricing, the decision history, has to live somewhere a model can read, in a shape a model can use. That is not a technology problem. It is a knowledge architecture problem. The single most valuable hour in many AI programs is the hour spent deciding which system is the source of truth for which fact, and then enforcing it.

3. Runway before rollout

Before the pilot ships, four things need to be true. An owner exists. A metric exists. A feedback loop exists. And a governance answer exists for what happens when the AI is wrong. Those are not dramatic requirements. They are the same things a junior hire would need on their first day. Most pilots ship without them because the team running the pilot is not the team that has to live with it, and the hand-off never actually happens.

These three principles are also the shape of Radiant Work's engagement model. The Operations Audit is the audit-before-automate step: two weeks, standardized scope, a clear picture of where friction is and what is ready to automate. The Architecture Sprint is the context-before-compute step: one week, focused on building the source of truth the rest of the system will depend on. The Implementation Sprint does the actual build, simple or complex. The Advisory Partnership is what keeps the system honest as the business changes, because the thing that breaks an AI system is never the code, it is the business quietly evolving around it.

None of this is a critique of pilots. Pilots are the right way to learn. The critique is of pilots run as one-offs, disconnected from the operation they were supposed to serve, shipped with no runway, and then blamed on the model when they fail. Any AI program that does not begin with the honest work of understanding the operation, consolidating the context, and designing the runway is going to end up in the 95%.

The MIT study was read as a failure report. It is more useful as a map. It tells you where the cliff is, what the people who fell off were doing, and what the people who didn't had in common. If you want a deeper view of how we think about this problem, our approach to AI operations walks through the ground-up version, and the frequently asked questions page covers the practical edges.