The AI app lifecycle and evals

Why a demo is not a product

Classic machine learning teams spend most of their effort training models. With LLM apps you usually rent a model as a service, so the real work moves to prompts, data, integration and checking the output. That shift has a name, LLMOps, and it borrows the discipline of software testing for something far less predictable than normal code.

Three things make LLM features fragile. The same input can produce different text on different runs. The provider can update or retire a model while your code stays the same. A tiny wording change in the system prompt can fix one case and quietly break five others.

The answer is not to stop changing things. It is to make every change measurable, so you know whether Study Buddy got better or worse instead of guessing from a few chats.

The lifecycle: ideate, build, evaluate, operate

Think of the lifecycle as loops, not a straight line. Each stage feeds the next, and what you learn in production flows back into the next round of ideas.

Around all four stages sits governance: who may see the data, which laws apply, how keys are stored and who signs off a release. Lessons 4 and 14 covered most of it, and it never goes away.

Do not skip the ideate stage because coding feels like progress. Ten minutes of trying the task by hand often shows that the idea needs retrieval, a tool or a narrower scope, and that is far cheaper to learn before you write the app around it.

• Ideate: write down the job in one sentence, try it by hand in a playground with two or three models, and collect the first real examples of input you expect.
• Build: turn the best prompt into code, add retrieval, tools or structured output where needed, and keep prompts in files under version control.
• Evaluate: run a fixed set of test cases with pass rules, compare versions side by side and decide with numbers.
• Operate: deploy, log every call, watch errors, latency and spending, collect user feedback and add new failures to the eval set.

Eval sets and regression tests for prompts

An eval case is an input plus a rule that says what a good answer must contain or avoid. Ten to thirty cases are enough to start, as long as they cover typical requests, edge cases and things the app must refuse.

Good cases come from real use. Start with questions you asked Study Buddy yourself, add the ones friends type when they try it, and write a few deliberately awkward inputs: a misspelled topic, a question in another language, a request for the answer key instead of a quiz. Store them in a JSON file next to the code so they are versioned together.

Pass rules come in four strengths. Code checks are the cheapest: does the answer contain a keyword, parse as JSON or stay under a length limit. A second model can grade answers against a written rubric, which handles open-ended text but needs spot checks by a person. Human review is the slowest and the most trustworthy, so keep it for a sample. Comparison tests show two versions of an answer and ask which one is better.

Treat the eval set as a regression suite. Every time you edit a prompt, swap the model or change retrieval, run the whole set and compare the pass count with the previous version. A failure that a user reports becomes a new case, and that case stays in the set for good.

Model output varies even at temperature 0, so a single run can mislead you. Run important cases several times and look at the pass rate, not one lucky answer. Change one thing at a time, otherwise you cannot tell which edit caused the difference.

What to measure

A single pass count hides too much. Track a few separate numbers so you can see which dimension moved after a change.

• Quality: does the answer do the job the user asked for, judged by your pass rules.
• Groundedness: when the app uses notes or documents, is every claim supported by them.
• Harm: does the app refuse what it should and avoid unsafe, biased or private content.
• Latency: seconds until the first token and until the full answer.
• Cost: input and output tokens per request, multiplied by your provider's rates.
• Failures: timeouts, invalid JSON, empty answers and provider errors per hundred calls.

Logging and cost in operation

Write one structured log line per model call. Include a timestamp, the prompt version, the model name, token counts, latency, whether the answer passed any automatic checks, and an id that links it to user feedback. JSON Lines files are fine for a small app; a database or a tracing tool helps later.

Logs can hold personal data, so decide what you store. Mask emails and names where you can, keep logs only as long as you need them, and tell users in plain words that conversations may be reviewed to improve the service.

Cost grows with tokens, and output tokens usually cost more than input tokens. Most savings come from boring moves: shorter system prompts, sending only the relevant notes, a sensible max_tokens limit, caching answers to repeated questions and routing easy tasks to a smaller model. Set a spending limit or alert in your provider's dashboard before anything goes public.

Close the loop with feedback. A thumbs up and thumbs down under each answer costs little to build, and every thumbs down is a candidate eval case. Review a handful of logged answers every week, because dashboards show that something changed, while reading answers shows what changed.