Choosing What to Build

Part 5 of my series on AI in production. Parts 3 and 4 were about making agents safe to run. This part steps back to an earlier question: which problems deserve an AI system at all?

Ask any leadership team for AI ideas and you'll get a long list within an hour. A chatbot for customers. An assistant for sales. Automated reports. Contract review. Something with agents.

Most of those ideas will never reach production. Some get built as impressive demos and then stall, because nobody owns the outcome or the data isn't there. Others run as pilots for months without anyone deciding whether to scale them or stop. Teams end up with a dozen half-finished experiments and nothing that changes the numbers.

The problem is rarely the technology. It's prioritisation. Ideas are chosen because they demo well, because a competitor announced something similar, or because a senior person asked for them. Choosing what to build with AI needs the same discipline as any other product decision, plus a few questions that are specific to AI.

Start from the problem, not the model

The best AI projects I've seen didn't start with "where can we use AI?" They started with "where are we losing time, money or customers?" and only then asked whether AI was the right fix.

Walk the workflow with the people who do the work. Where do they wait? Where do they search for information? Where do they repeat the same judgement many times a day? Where do mistakes cost the most? A list built this way is grounded in real pain, with a real owner, before any technology is chosen.

Then check whether each problem is AI-shaped. The strongest candidates usually share a few traits:

• High volume. The task happens hundreds or thousands of times, so small improvements add up.
• Language-heavy. It involves reading, writing, searching or summarising text, where today's models are strongest.
• Judgement within known rules. Experts can explain what a good result looks like, even if it takes them time to produce one.
• Mistakes are catchable. A person can check the output quickly, or an error is cheap to fix.
• The data exists. The documents, records and history needed to do the task are available and permitted for use.

If a problem fails most of these, a simpler fix often wins: a better form, a clearer process, a rule in the CRM. Not every problem needs a model, and choosing the simpler option is a sign of good judgement, not a lack of ambition.

Score on value and feasibility, then filter on risk

Once you have a shortlist of AI-shaped problems, score each on two things. Value: how much revenue, cost or customer experience moves if it works, owned by a named business leader. Feasibility today: whether the data, the integration points and a clear definition of good already exist.

The top-right corner is where to start. These projects prove value quickly, and they build the shared pieces, such as retrieval, logging and evals, that make the next project cheaper. Big bets deserve funding too, but one at a time, with milestones that show whether feasibility is improving.

Then apply risk as a filter, not a score. A use case with high value and high feasibility can still be wrong to start with if a single mistake reaches customers, moves money or breaks a regulation. That doesn't rule it out. It means it starts as an assistant that a person checks, as described in part 4, rather than an agent that acts alone.

The feasibility questions to ask before funding

Feasibility is where AI projects are most often misjudged, because a demo can make anything look feasible. Before committing a team, I ask these questions, and a "no" to any of them is a reason to fix that first or pick another problem.

The last question is the one most often skipped. An AI project without a business owner who wants it is a technology project looking for a customer.

Sequence for learning, not just for value

The order you build things in matters as much as what you choose. A few principles have served me well:

• Start with one workflow and one team. A narrow launch with a team that wants it beats a broad launch to people who were told to use it. You learn faster, and you get a real result to show.
• Assist before you automate. The first version drafts, suggests and summarises while a person decides. It earns trust and produces the approval data that later justifies more autonomy.
• Build the shared layer once. The first project should leave behind pieces the next one reuses: connections to the CRM and documents, logging, the eval process, the approval patterns. That's how the second use case ships in weeks instead of months.
• Run a portfolio, not a single bet. Most of the effort goes to quick, proven use cases that fund the work. A smaller share goes to one bigger bet that could change the business if it works.

Platform thinking pays off here. When I built open APIs and developer platforms, the first integration was always the slowest; every one after it reused what the first had built. AI capabilities work the same way. Judge the first project partly on what it makes possible next.

Saying no, and stopping on time

Prioritisation only works if some ideas are turned down and some pilots are stopped. Both are harder with AI, because every idea sounds strategic and every pilot shows promise in a demo.

• Time-box every pilot. Set a decision date and the measure that will decide it before the pilot starts. On that date, the choice is scale, change or stop, never "keep exploring."
• Agree kill criteria up front. For example: the eval pass rate is still below the launch bar after two iterations, or the target team isn't using it after a month.
• Say no with a reason. "Not now, because the data isn't ready" or "not now, because the value is small" keeps the idea on the list and the relationship intact. It also shows the requester what would change the answer.
• Count experiments, not features. The roadmap for AI work should show which questions you're testing and what you learned, not just what you shipped.

Stopping a pilot that didn't work isn't a failure. Keeping one alive for a year because nobody wanted to decide is.

Strategy is what you don't build

The pressure to "do something with AI" is real, and it pushes teams towards many small starts and few finishes. The teams that get value do the opposite. They choose a small number of problems that matter, check that they're feasible before committing, start with a narrow version that people want to use, and stop what isn't working.

None of this is new. It's ordinary product strategy applied to a new technology. What's different is how convincing a demo can be, which makes discipline more important, not less.

Next in the series: the trade-offs behind every AI feature. Model quality, cost and speed, and how to decide which one to give up.