AI Answer Library
Short answer
Four steps: pick one narrowly scoped use case, build an evaluation set, ship a minimum usable version, and only then discuss rollout. The goal of a first project is not an impressive system but an organisation that has learned how to judge one. Without an evaluation set there is no basis for judgement, and every later debate collapses into "it does not feel right". Choose a use case with real users where failure does not touch the core business — do not practise on a critical process.
The table expands those four steps a little further — "rollout" actually contains two distinct stages, a limited live trial and an operations handover, and plenty of projects die in exactly those two. What matters is not the phase list but that each phase has an explicit test for moving on; phases without exit criteria merely slice one blob of work into several and reduce no risk. Put the last column into the project plan at kickoff, and agree that an unmet criterion means slipping the date rather than skipping the gate.
| Phase | Goal of the phase | Deliverable | Exit criterion |
|---|---|---|---|
| 1. Use-case selection | Find a bounded problem that real people actually have | A one-page brief: who uses it, how often, and how it is done today | You can state what happens if nothing is built, and a named business owner exists |
| 2. Evaluation set | Turn "is it good" into something measurable | Real questions, reference answers and a scoring rubric, authored by the business side | Two people scoring independently under the rubric reach broadly the same result |
| 3. Minimum usable version | Get the entire chain working with the plainest possible approach | A system real users can open and operate themselves | Evaluation-set performance has converged instead of swinging with every change |
| 4. Limited live trial | Surface the problems the evaluation set cannot cover | Usage logs, a failure-case list and recorded user feedback | Someone keeps using it without being told to |
| 5. Rollout and handover | Make your own team able to maintain and evolve it | An admin console, operations documentation and a real handover session | An in-house engineer ships one change end to end without help |
Choosing the use case matters far more than choosing the technology, and it is where teams most often go wrong. Three failure patterns recur. First, picking something that looks valuable but nobody uses daily — an analytics assistant for executives, say. Low frequency means no feedback, and the project quietly stops at the demo stage. Second, picking a core business process: once mistakes affect production, the team adds more and more manual review out of caution until the automation benefit reaches zero. Third, leaving scope open, as in "we want it to answer anything" — such projects always have someone dissatisfied at acceptance. A workable filter requires three things simultaneously: a fixed group hits this problem every week; the current way of doing it can be described precisely; and if the output is wrong, someone downstream will notice and correct it. Use cases meeting all three are rarely impressive, but they succeed far more often — and what a first project needs most is to succeed once.
Three things must be in place before kickoff. First, a business owner with decision authority, not merely a technical contact. AI projects continually raise judgement calls about how a given situation should be answered, and with nobody empowered to settle them the project stalls in endless discussion. Second, clear ownership of the data: who decides which documents enter the knowledge base, which version is authoritative, and who may see what. These are technically trivial and organisationally hard, and solving them early saves the most time. Third, calibrate expectations explicitly: the system will make mistakes, version one will be worse than a human, and the value comes from absorbing a large volume of easy cases rather than replacing experts. Skip this and the first visible error becomes "this thing does not work". None of the three requires technical investment, yet their absence is the most common cause of failure — and no amount of technical strength compensates for it.
Where this applies
People also ask
How do I choose an enterprise AI solution provider?
How long does it take to deploy a self-hosted large language model?
How accurate can an enterprise RAG knowledge base actually be?