Seven bugs in sixteen hours, or why AI pilots fail
95% of generative AI pilots deliver no measurable return, according to MIT's 2025 State of AI in Business report.
Seven bugs in sixteen hours should have made this pilot part of that statistic. It didn't, and the reason why says more about AI pilots than the failure rate does.
The pilot trap
This month, a team of mine ran an end-to-end load test under real production conditions on a multi-agent development pipeline, with real tasks and real handoffs between agents. Within sixteen hours it had found seven bugs. I won’t bore you with the details, but none of these threw an exception and all of them would have quietly corrupted a production run.
By most people's definition, that is exactly what a failed pilot looks like. Seven bugs on day one is the number companies point to when they decide to scrap the programme, write off the spend, and go back to the original process. Most teams see a wall of failures in the first real run and conclude the technology was not ready.
They are optimising for the wrong signal. A pilot that reveals nothing wrong on its first real run under load has not been tested, it has been demoed. The absence of failure at that stage is not evidence of readiness, it is evidence that nobody looked hard enough.
Breaking it on purpose, safely
All seven of those bugs were fixed within the same working day. Not because the fixes were trivial, but because every action in that pipeline was evented and logged from the start, so each failure was traceable to a single point and reproducible on demand. Nobody spent the afternoon guessing which agent did what. The system told them.
That is the real world difference between the 5% of pilots that scale and the 95% that stall. It has nothing to do with model quality or how clever the agents are, it comes down to whether the team built the observability that turns a silent failure into a loud, specific, fixable one. Speed without that instrumentation results in a possibly, no, probably fragile system.
Ensuring we have the observability in place before a customer ever sees it is the only option. Scail's AI Risk Value Index is built around exactly this distinction, separating organisations that can see what their AI systems are actually doing from organisations that are only watching the output.
What boards need to see now
Ask any board member which of their AI pilots are working and most will point to a demo, a slide, or a pilot metric measured in a controlled environment. Very few can tell you what happened the first time the system ran under real conditions, because very few systems are instrumented to say.
That gap is the real risk. Not the bugs themselves, the inability to see them. The Scail AI Risk & Value Scorecard assesses AI capability across eight areas, from governance and risk through to execution and value realisation, and asks the harder question underneath all of them, which is whether the organisation can actually observe what its AI is doing while it does it.
The regulated SaaS businesses that beat that 95% failure rate will not be the ones with the smoothest pilots. They will be the ones that built the instrumentation to survive an ugly first run, then used everything it revealed.
Read more about our AI Risk & Value Scorecard.
