Getting a convincing LLM demo takes an afternoon. Getting something you can put in front of students, bank staff or compliance officers takes considerably longer, and the gap is almost entirely about what happens when the model is wrong.
Decide what the model is allowed to be wrong about
Before any architecture, write down which outputs are advisory and which are consequential. A suggested study topic can be wrong; a compliance classification or a payment instruction cannot. Those two categories need different treatment, and conflating them is the most common design mistake we see.
For consequential outputs, the model should propose and a deterministic system should dispose — validation, rules, or a human review step that is part of the product rather than a disclaimer at the bottom of the screen.
Ground answers in your own data
A general model knows general things. It does not know your syllabus, your product catalogue or the regulation you operate under. Retrieval over your own content is what turns a plausible answer into a correct one, and it has a useful side effect: you can show the source alongside the answer.
That citation is not a nicety. It is how a user decides whether to trust a given response, and it is how your support team debugs a complaint six weeks later.
Budget for latency and cost as product constraints
Model calls are slow and metered. Both are product decisions, not infrastructure details. If a feature needs a response in under a second, a large model in the request path will not work, and no amount of optimisation will change that — you need a smaller model, a cache, or a different interaction design that sets the expectation.
The same applies to cost. A feature that costs a few paise per call is fine until it is on a screen a hundred thousand users open daily.
Log the inputs, not just the outputs
When a user reports a bad answer, you need the prompt, the retrieved context and the model version to reproduce it. Teams routinely log the response and discover they cannot explain it. Treat an LLM call like any other external dependency: record what went in, what came back, and which version answered.
Evaluate on your own examples
Public benchmarks tell you very little about whether a model works for your use case. A set of fifty real examples from your own domain, with the answers you would accept, is worth more than any leaderboard — and it is what lets you switch models later without guessing whether you have made things worse.
The short version
Treat the model as a component with a known error rate rather than an oracle. Design the product so that being wrong is survivable, visible and recoverable, and the engineering around it stops being frightening.
