Why most AI pilots stall, and the four questions that get them into production

Most firms now have an AI pilot or two. Far fewer have one running in production with the controls their clients and regulators expect. The gap usually comes down to four questions that were never asked before the build began.

The pattern behind a stalled pilot

Typically, a pilot starts with energy. A team sees what a model can do, builds something impressive in a few weeks, and shows it to leadership. The demo lands well. Then progress slows. Risk wants to understand how outputs are checked. Finance asks what it will save. Operations asks who will run it. Nobody can answer with confidence, so the pilot waits.

Having taken AI-assisted document extraction from prototype to live service inside a regulated RegTech business, we see the same thing again and again. The pilots that reach production answered four questions at the start.

1. Which decision or task are we changing?

A pilot framed as "use AI on our documents" has no edge to it. A pilot framed as "reduce the time an analyst spends extracting fund terms from a prospectus" does. Naming the specific task tells you who the users are, what good output looks like and what a mistake would cost.

It also makes the next three questions answerable. If you cannot describe the task in a sentence your operations lead would recognise, the pilot is still an idea.

2. What is the baseline, and who agrees it?

Value claims fall apart when nobody measured the starting point. Before building anything, capture how long the task takes today, how often it goes wrong and what rework it creates. Agree those numbers with the people who will judge the result, ideally including someone from finance.

With an agreed baseline, the end of the pilot becomes a straightforward measurement that everyone already trusts.

3. Where does a person review the output?

In regulated work, the question regulators and clients ask is simple: how do you know the output is right? Decide early where a person reviews the model's work, what they check, and what happens when they disagree. Record those reviews. That record becomes your evidence that the service is controlled.

Designing human review into the workflow from day one is far easier than adding it after a pilot has been shown to the board.

4. Who owns it once the pilot ends?

Pilots are often run by enthusiasts, and production services need owners. Agree who will own the service, who maintains the prompts and models, who monitors quality and who signs off changes. If that person is not in the room during the pilot, the handover will be hard.

Putting the four questions to work

These questions sit at the heart of our approach to AI adoption: map the task, score it, prove it against an agreed baseline and embed it with clear ownership and controls. They work equally well for a single use case or a portfolio of ideas.

• Before you build: write a one-page brief that answers all four questions.
• During the pilot: measure against the baseline and log every human review.
• At the decision point: present the measured result, the control design and the named owner together.

A pilot that arrives at the decision point with those three things is ready for a yes. If you would like a second pair of eyes on a pilot that has stalled, our AI adoption service starts with exactly this conversation.