The gap between an AI demo and an AI product is where most projects quietly die. I've seen it enough times to recognise the pattern within the first two weeks of any engagement: an impressive prototype, a team that's excited, a business case that looks compelling on a slide — and an invisible cliff somewhere between "look what it can do" and "this is live in production, reliably, serving real users."
Gartner put a number on it: 87% of AI projects never make it to production. In my experience, that figure tracks. I've rebuilt four projects that had stalled at the cliff edge. Here's what was different about the four that shipped.
Why 87% Die (It's Not the Model)
The failure mode is almost never technical model quality. The demos that die are often genuinely impressive — they use good models, they produce plausible outputs, the data scientists are skilled. They die for reasons that have nothing to do with the ML:
No production serving infrastructure. The demo runs on a data scientist's laptop or a Jupyter notebook. There is no API, no container, no latency budget, no error handling, no authentication. Getting from notebook to a reliable endpoint that a product team can call is a 3–6 month engineering project that nobody planned for.
No monitoring. Models degrade in production — data drift, distribution shift, upstream data pipeline changes. Without monitoring, you don't know the model is wrong until users complain, customers churn, or a regulator asks why your fraud model stopped catching anything. Most AI projects allocate zero headcount to monitoring.
No retraining loop. The model was trained on data from last year. Production data moves. Without a retraining pipeline, you have a static artefact decaying in real-time. This is the norm, not the exception.
Ownership is unclear. Data science built it. Engineering didn't. Operations won't touch it. The project lives in a gap between team boundaries and nobody has the mandate or the budget to close it.
The business case was built on demo performance. Accuracy on a curated test set looks very different from accuracy on the messy, edge-case-heavy, seasonally-shifting data that production actually serves. The business case collapses when the real-world number is 20 points lower than the demo number.
The demo-to-production gap is an organisational problem, not a technical one
The single most common root cause I find is that the AI project was owned by a data science team with no engineering mandate, and the engineering team treated the output as someone else's problem. Shipping an AI product requires a decision that it is an engineering deliverable — with production SLAs, on-call ownership, deployment pipelines, and monitoring. That decision is almost always a leadership decision, not a technical one.
Four Projects That Died. What I Found.
Here are the four stalled projects I was brought in to rebuild, with the core failure mode that had killed each one.
Project 1: Insurance Claims Triage Model (Major Reinsurer)
What they had: A gradient boosting classifier that predicted which claims required manual review, trained on 3 years of claims data. Accuracy on held-out test data: 91%. The demo had been running for 8 months. It had never handled a single production claim.
What killed it: No serving infrastructure and no agreement on who owned it. The data science team had handed off a pickle file and a Jupyter notebook. The engineering team had no framework for deploying ML models. There was no container, no monitoring, no retraining schedule. Eight months of back-and-forth between teams, with nobody empowered to just build the missing layer.
What we did: Built a FastAPI serving layer, containerised with Docker, deployed on Kubernetes with health checks and circuit breakers. Added Evidently for drift monitoring with alerts routed to a shared Slack channel. Established a monthly retraining schedule triggered by drift metrics. Claims triage went live 11 weeks after I joined.
The business outcome: 34% reduction in manual review workload for the claims operations team, with no measurable change in accuracy from the original demo model.
Project 2: Customer Churn Prediction (Insurtech Scaleup)
What they had: An LSTM model trained on customer behaviour sequences. It worked beautifully in notebook form. Product management had been waiting 14 months for it to ship.
What killed it: The upstream data pipeline was owned by a different team and had no SLA. The model consumed features from 6 different data sources. On any given day, 1–3 of those sources had missing or delayed data, and the notebook silently produced wrong predictions without any error. Nobody caught it because nobody was watching.
What we did: Rebuilt the feature pipeline with explicit data quality checks using Great Expectations. Added alerting when upstream data was late or out of range. Decoupled the serving pipeline from the training pipeline so a retraining failure wouldn't take down production predictions. Added a fallback to a simpler rule-based model when data quality checks failed.
The business outcome: Churn prediction model in production within 8 weeks. Customer success team now receives daily high-risk customer lists. Retention campaign targeting improved by 28% in the first quarter post-launch.
Project 3: Document Classification (Legal Firm's Contract Review)
What they had: A fine-tuned BERT model for classifying contract clauses into 22 categories. Test accuracy: 94%. It had never touched a real contract outside the test set.
What killed it: The test set was beautiful and clean. Production contracts were PDFs with tables, footnotes, scanned pages, non-standard formatting, multi-language clauses, and edge cases the training data had never seen. Real-world accuracy on production contracts was closer to 71%. Nobody had tested it on real production documents before building the business case.
What we did: Ran a two-week production shadow mode — the model processed real contracts in parallel with human reviewers, with no output to users. Identified the failure categories. Added a pre-processing layer to handle PDF extraction edge cases. Retrained on a sample of real production documents. Built a human-in-the-loop review queue for low-confidence predictions rather than hard automation.
The business outcome: Model in production at 88% accuracy on real contracts, with a confidence-gated human review queue handling the remaining 12%. Contract review time down 41% for the covered clause categories.
Project 4: Energy Demand Forecasting (Utility Group)
What they had: An ARIMA-based demand forecasting model that outperformed the incumbent vendor model in backtesting. The vendor contract was up for renewal. The internal model had been ready for 10 months.
What killed it: No mechanism to deploy the model into the operational workflow. The demand forecasting output fed into 4 downstream systems operated by 4 different teams. Each team had different data format requirements, different update cadences, and different integration points. The data science team had built the model but had no access to those systems and no organisational authority to drive the integration.
What we did: Mapped the downstream integration requirements first, before touching the model. Built a standardised output adapter layer that transformed model output into each downstream system's required format. Got executive sponsorship to give us cross-team access. Ran a 6-week parallel run alongside the vendor model before full cutover.
The business outcome: Internal model replaced the vendor contract, saving €380K per year in licensing fees. Forecast accuracy improved by 12% MAPE against the vendor's model on out-of-sample data.
The Pattern That Separates Ships from Stalls
Across all four, the fix shared the same structure:
| What failed | What we fixed |
|---|---|
| No production serving layer | Containerised API with health checks, circuit breakers, load testing |
| No monitoring | Drift detection, prediction logging, data quality alerting |
| Broken upstream data | Explicit data quality gates, fallback models, pipeline SLAs |
| Team ownership gap | Named product owner, shared on-call, engineering squad formally assigned |
| Business case built on demo performance | Shadow mode on real data before launch, realistic accuracy benchmarks |
| No retraining loop | Automated retraining triggers on drift metrics, staged rollout with canary |
None of these fixes required a better model. The model was almost always fine. What was missing was everything around the model — the infrastructure, the governance, the operational discipline.
The full stack that every shipped AI project has in place looks like this:
The question that diagnoses a stalled AI project in 10 minutes
Ask the team: "If the model's predictions degraded by 15 percentage points tomorrow, how long would it take you to notice?" If the answer is anything other than "immediately, via an automated alert," the project is not production-ready — regardless of what the demo looks like.
What You Should Demand From Your AI Team
Before the next AI project goes into development, I'd push for answers to these questions:
Serving: How will the model be called in production? What is the latency budget? What happens if it throws an error? Who manages the serving infrastructure?
Monitoring: How will you detect data drift? Model performance degradation? What triggers a retraining run? Who gets paged when something breaks?
Data quality: What upstream data does the model depend on? What is the SLA on each upstream source? What does the model do when a source is missing or late?
Ownership: Which engineering team owns this in production? Who is on call? Who has the mandate and budget to maintain and improve it?
Realistic benchmarking: Has this been tested on actual production data, not a curated held-out set? What is the accuracy on the messiest 10% of real inputs?
If you can't answer all five — confidently, with named people and concrete numbers — you don't have an AI product. You have a demo with a delivery date problem.
The 13% that ship aren't smarter or luckier. They made the platform decisions early, assigned engineering ownership before the model was built, and tested against reality instead of a clean benchmark. That's repeatable. Every time.
If your AI project is stalled between the demo and the product — or if you're building one and want to avoid the cliff — let's talk. I work with teams to design the production infrastructure alongside the model, not after it. Book a 30-minute discovery call.