Almost every company we talk to at Nortik has the same artifact sitting in a repo somewhere: an impressive AI prototype that has been "two weeks from launch" for six months. It nailed the demo. Leadership loved it. Then it quietly stalled.
This is not a story about bad engineering. The people who built those prototypes did exactly what a prototype is for. They proved the idea works. The stall happens because getting from "works in a demo" to "works in production" is a different discipline, and most teams only find that out after the applause dies down.
So here is what that distance is actually made of, and how to cross it on purpose instead of stumbling into it.
The demo was the easy part
A demo runs on the happy path. The person presenting picked the inputs, the audience saw three or four runs, and nobody measured latency or cost because nobody cared yet. Under those conditions modern models look magical, which is exactly why a good demo creates a false sense of being done.
Production flips every one of those conditions:
- You don't choose the inputs anymore. Users paste in broken HTML, emails that are half Serbian and half English, 40-page PDFs, and questions the system was never meant to answer. The prompt that handled your ten test cases meets an input distribution it has never seen, on day one.
- Nobody is watching each output. In the demo, a human implicitly reviewed every response. In production, output number 4,000 goes to a customer without anyone looking at it. The question stops being "can it do this?" and becomes "what is the worst thing it will do this week?"
- Cost and latency become product features. A 30-second, $0.40 response is fine in a demo and fatal in a checkout flow. Most prototypes have never been profiled at all. The number is knowable early: the eleven-page report Vinready renders in about six seconds only happens because latency was a requirement, not a postmortem.
None of this means the prototype was wasted. It answered the only question it could answer: is this worth engineering properly? If the answer is yes, the real work starts there. It doesn't end there.
Real inputs break prompts that demos never test
The most common failure we see in rescue projects is a system whose behavior was defined by example rather than by specification. The prompt grew organically during demo season. A sentence added here to fix yesterday's bug, an example appended there, until it became a load-bearing artifact nobody fully understands.
That approach collapses the first time you need to change anything, because there is no way to know what a change breaks. You fix the formatting issue a customer reported, and three days later you find out the fix also changed how the system handles refunds. Without a test harness, every prompt edit is a blind bet.
The fix is boring and it works: treat every behavior you care about as a written-down, testable requirement. "Never quote a price that isn't in the source document." "Always answer in the user's language." "Refuse medical advice and route to a human." Each of these becomes a test case, which brings us to evals.
Evals are the unit tests of AI engineering
If one practice separates teams who ship from teams who stall, it's this one. An eval suite is a set of representative inputs, real ones pulled from logs where possible, with an automated way to score the outputs. It answers the question every stalled team is quietly afraid of: if we change anything, how do we know we didn't make it worse?
A workable eval setup is smaller than most teams expect:
- Start with 50 to 100 real cases, not 10,000 synthetic ones. Pull the inputs users actually sent, including the ugly ones. Twenty representative failures are worth more than a thousand generated successes.
- Score with code where you can, with a judge model where you can't. Format checks, factual-grounding checks, and forbidden-content checks are plain assertions. Subjective quality like tone and helpfulness can be scored by a second model against a written rubric.
- Run the suite on every change. Prompt edits, model upgrades, retrieval changes, all of it goes through the evals, in CI, like any other test suite. A model version bump stops being a leap of faith and becomes a diff you can read.
- Grow the suite from production. Every incident becomes a new eval case, the same way a bug becomes a regression test. Six months in, your eval suite is the most valuable AI asset you own. It encodes what "good" means for your product, independent of any model vendor.
Production AI is an infrastructure problem, not a prompt problem
People usually describe the prototype-to-production gap as a model quality problem. In our experience it's mostly an infrastructure problem, the unglamorous layer around the model call that nobody scoped during demo season:
- Failure handling. Providers have outages, requests time out, models return malformed JSON. Production systems need retries with backoff, a degraded mode, and a real answer to "what does the user see when the model fails?" Because it will fail.
- Observability. When a customer reports a bad answer from last Tuesday, you need the full trace: input, retrieved context, prompt version, model version, output, cost, latency. If you can't replay a bad output, you can't fix it.
- Cost and latency budgets. Per-feature budgets, enforced with caching, model routing (small model for easy cases, large model for hard ones), and truncation strategies. We've cut inference bills by 60 to 80% on systems that had simply never been profiled.
- Guardrails at the boundaries. Input validation before the model, output validation after it. Schema checks, grounding checks, PII filters. Cheap, deterministic code that turns "the model probably won't" into "the system can't."
This part looks like classic backend engineering because it is. It's also why AI engineering as a discipline sits closer to production software than to data science. What separates the teams that ship isn't prompt cleverness, it's whether the system around the model is built to operate.
How to take a prototype to production in six weeks
The teams we've watched cross the gap fastest all follow roughly the same sequence. It's the one we run at Nortik as a standard engagement:
- Week 1: define "correct" in writing. Turn the prototype's implicit behavior into explicit requirements and collect 50 or more real inputs. This week produces the first version of the eval suite, and it usually surfaces the disagreements about what the feature is supposed to do that were hiding behind the demo.
- Weeks 2 and 3: build the harness, then touch the prompts. Stand up tracing, run the evals, get a baseline score. Only then iterate on prompts, retrieval, and model choice, with every change measured against the baseline instead of vibes.
- Weeks 4 and 5: build the failure paths. Retries, fallbacks, guardrails, budgets, the degraded mode. Load-test at realistic concurrency. This is where the 30-second response gets found and fixed, not in the postmortem.
- Week 6: ship behind a rollout, not a launch. Five percent of traffic, tracing on, evals running against production samples daily. Expand as the numbers hold. The launch announcement comes after the system has been boringly stable for a while, which is the only kind of stable that counts.
Six weeks is not a law of nature. A complex agentic system takes longer, and a well-scoped extraction feature can be faster. The sequence is the point: specification, then measurement, then hardening, then gradual rollout, each step making the next one safe.
The prototype is a hypothesis, not a product
The healthiest way to think about it: a prototype is a validated hypothesis. Validating it was real work and real progress, but a hypothesis doesn't ship. Lumoo is the clean example: the Lovable MVP proved the idea, and the production build that followed is what Nordic fashion brands actually use. What ships is a system that holds that hypothesis up under load, under adversarial inputs, under provider outages, and under the scrutiny of a customer who doesn't know or care that there's a model behind the button.
That system can be built on a schedule, with practices that are neither exotic nor expensive. Written specs, eval suites, tracing, guardrails, gradual rollouts. The teams that stall aren't short on talent. They just haven't recognized that this second phase exists and deserves its own plan.
If your prototype has been "two weeks from launch" for a quarter, that plan is what's missing. It's exactly the work we do at Nortik, and it's very fixable.
