Production AI vs demo-ware

The room liked the demo. The prompt was clean. The answer sounded like a person who had read the brief. Someone asked when it could be “rolled out.” Nobody asked which table it read, who is allowed to say the answer is wrong, or what happens when last night’s load fails.

That gap is the whole product. Demo-ware is designed to look decided. Production AI is designed to keep being right — or to fail closed — after the slide deck leaves the room.

What the demo is hiding

A demo selects the question, the data, and the mood. It is a performance. Useful as a sketch. Dangerous as a contract. You are still in demo land when any of these are true:

• The answer cannot cite a table, a grain, and a time the data was fresh.
• A prompt tweak is how you “fix” a wrong number.
• There is no frozen set of questions it must still pass next week.
• The people who own the metric have not signed off on the output.
• Nobody can run it besides the person who built the notebook.
• The monthly running cost is a shrug.

If two extracts already disagree, an agent grounded on both of them will just argue faster. That is not production AI. That is a chatbot wrapped around the meeting you already hate.

What production actually requires

Production AI, on real operations work, is boring on purpose. It needs four things before a model choice matters:

1. A warehouse people already trust. Same official tables the reports use — not a folder of PDFs and a spreadsheet someone uploaded last week.
2. One job, measured. Invoice match, ticket draft, a report people hate writing. Hours saved or errors avoided, written down before you start. Not “AI across the business.”
3. An exam it has to pass. A frozen set of real questions and known-bad cases. If the score drops later, it does not ship.
4. A named human gate. AI drafts. Someone you name signs off. Access follows the same permissions the rest of the system already has.

Cost sits next to those four. Monthly running cost before you commit, and a hard stop if it does not earn its keep. A vendor promise with no cap is not an engagement. It is a subscription to hope.

Warehouse first, then a job, then a model

Teams reverse this. They pick a model, then hunt for a use case, then notice the numbers are a mess. The honest sequence is the other way around.

If Finance and Operations cannot agree what “revenue” means, the first delivery is a definition and a certified table — not a chatbot. One number everyone trusts is that work. Production AI comes after the number is real, and only if there is a job a person currently does that the model can take a measured slice of.

Grounding on the warehouse is not a slogan. It means: when the overnight load updates, the assistant sees the same refresh as the dashboard. If a figure is not in the official data, it is not in the answer. Every answer can point back to which table produced it.

When we will not build it

We will tell you not to build when:

• The source of truth is still two workbooks and a verbal rule.
• There is no owner for the output — only a steering committee.
• The “job” is a domain (“customer experience”) rather than a task with a metric.
• Customer names, accounts, or secrets would have to leave the building to make the demo work.
• Success is “people are impressed,” not hours saved or errors avoided.

That is not a moral pose. It is cheaper than a six-month pilot that cannot be handed over. When not to build is the scoping note. This one is the AI-shaped version of the same honesty.

What AKAF will do instead

Pick one job. Point at the tables. Write the exam. Cap the bill. Ship something an operations team can run — or stop. AI that ships is that bar. If the warehouse is not ready, data engineering is the first move, not a chatbot on a PDF pile.

Send the job, the system of record, and the decision it is supposed to change. We will tell you whether the work is production AI, a pipeline, or not yet.

Start a conversation →