Without clear links to customer value, most pilots fail to deliver. Disciplined focus can change that.
A recent MIT study found that 95 percent of investments in generative AI have produced no measurable returns. It’s a sobering figure, especially for leaders who have spent months urging their teams to experiment with this new technology. If nearly all those pilots failed, should companies stop trying altogether? Or is the problem not with experimentation itself, but with the way it’s done?
That question was at the heart of MIT’s study, which examined more than 300 public AI deployments, surveyed 350 employees and interviewed 150 senior executives. The researchers found that while individuals are adopting AI tools that make them more productive, most companies have yet to see real gains at the profit-and-loss level. And most spending goes into sales and marketing pilots, even though the biggest returns typically come from back-end transformations.
Such headlines have triggered a familiar wave of disillusionment, much like the one that followed the digital transformation boom a decade ago. Back then, many executives encouraged experimentation but let it run unchecked – a “10,000-flowers” approach in which they hoped a few lucky bets would blossom into breakthroughs. Instead, they ended up with scattered projects that drained time, talent and money. As researchers who study AI and teach about technological change, we see leaders repeating the same mistake today with AI: there is abundant enthusiasm but little focus on real business value.
When good intentions go wrong
Encouraging experimentation isn’t the mistake; allowing it to lose focus is. Without a connection to real business opportunities, such as rethinking how a company serves existing and new customers, experiments are bound to disappoint.
It’s tempting to frame AI as something revolutionary and world-altering. But in doing so, leaders often lose sight of business’s simplest truth: Its purpose is to solve problems for customers. AI should serve that mission, not distract from it.
Escaping this experimentation trap requires a disciplined process. First, leaders need to place AI within the broader arc of technological transformation. Second, they must focus on AI’s potential to help serve customers better. Third, they should experiment with a targeted set of opportunities designed for scalability. And finally, once value is proven, they must scale up systematically.
Step back to see the bigger picture
While AI dominates today’s headlines, it is part of a longer shift that has been unfolding for years. Businesses are moving from digital technology operating at the periphery – think IT departments managing hardware and databases – to digital at the very core.
In this new model, organisations are built around digital workflows and customer journeys rather than internal production lines. Humans still oversee decisions, but the engine of execution increasingly relies on data and algorithms. Ant Financial’s use of algorithms to assess creditworthiness, or Amazon’s dynamic pricing systems, shows what this evolution looks like in practice.
Understanding that bigger picture reminds leaders that AI is just one tool among many for transforming the organisation. The internet changed how customers were served but not why. It will be the same with AI.
Keep your eye on the customer
Seeing AI through the lens of customer value helps managers focus their efforts in a world where there is no single playbook. Experimentation is necessary to discover how AI can create value in each business. Yet when uncertainty dominates, it’s easy for leaders to revert to the “10,000-flowers” approach, scattering resources across a garden of small, disconnected trials.
Your company doesn’t need to mimic a tech giant. You don’t need to become Meta or Google, nor chase viral headlines about firms firing most of their workforce to “go all in on AI”. The real opportunity lies closer to home. You should examine your internal operations and customer journeys to pinpoint where AI can create tangible value today.
By creating immediate value, focused projects become the Trojan horse for AI adoption. Far from trickery, they open the door to broader transformation.
Run smart, not scattered experiments
Once priorities are clear, the next challenge is designing experiments that actually teach the organisation something. That means three things. First, connect experiments to genuine value creation. Second, keep costs low to allow multiple learning cycles. Third, design each initiative with future scalability in mind.
This balance is tricky. Some leaders rush ahead with pilots that can’t scale. Others get paralysed by overengineering enterprise-ready systems from day one. The goal is to hit the middle ground – solve a meaningful problem at small scale, but with a clear path to scaling if it works. To decide which ideas deserve attention, we often use the IFD framework:
- Intensity (how severe the problem is),
- Frequency (how often it occurs) and
- Density (how many people experience it).
A baby monitor that detects breathing issues scores high on all three dimensions; a scheduling tool for apartment managers doesn’t. This approach helps leaders prioritise experiments with the best odds of real impact.
Scale with purpose and the right team
Showing value is only half the journey; scaling it demands structure and leadership. As organisations expand from pilot to enterprise level, new challenges inevitably arise across data integration, governance, ownership or culture.
Some organisations solve this by creating small, empowered task forces, which we call ninja teams, groups with both the mandate and the resources to drive change. At Amazon, Qualtrics and 7-Eleven, such teams benefited from senior-level sponsorship, cross-organisational connections and the autonomy to iterate quickly.
Once an initiative gains traction, the process can repeat – experiment, validate, scale – but always grounded in the same question: How does this help us serve customers better?
Don’t mistake failure for futility
As the hype around AI settles, many executives risk drawing the wrong conclusion from early disappointments. Some are already deciding that AI doesn’t work, just as they once declared digital transformation a fad. Yet in both cases, the issue isn’t the technology’s potential but the discipline with which it’s applied.
AI is already generating real gains, particularly in complex back-end operations where multi-agent systems are automating and optimising tasks. Remember, however, that lasting success always begins with the design of the first experiment, the moment when a team asks how this new tool can solve a meaningful problem for a real customer.
Technology will evolve, but that purpose will not. The companies that combine curiosity with discipline, and never lose sight of the customer, will be the ones that turn experimentation into transformation rather than waste.
