Before you ship an LLM feature: a checklist

    The demo works, the team is pleased, and someone asks when it can go live. Before it does, it’s worth running through a short list. None of this is exotic, but each item is something that has bitten teams after launch.

    The checklist

    Figure 1 · A pre-launch checklist

    AreaThe questionDone when
    QualityHow good is it on real inputs?A fixed test set exists and a pass bar is agreed
    SafetyWhat if someone feeds it hostile text?Permissions are minimal and risky actions need approval
    PrivacyWhat personal data does it touch?You know where prompts and logs are stored and for how long
    CostWhat does one request cost, and what is the cap?There is a per-request estimate and a limit on runaway use
    FailureWhat happens when the model is slow or wrong?Timeouts, fallbacks and a human handover are in place
    MonitoringHow will you know it’s getting worse?Logs, user feedback and a regular review exist
    OwnershipWho is responsible after launch?A named person owns it
    Adapt the list to your product. The right-hand column is what we would accept as finished.

    The ones that get skipped

    Cost. OWASP’s 2025 list for LLM applications includes unbounded consumption, which covers runaway API costs as well as denial of service. One long loop or one abusive user can run up a bill. Set limits before launch, not after the first surprise.

    Quality. “It looked fine in the demo” isn’t a test. Use real inputs, including awkward ones, and keep the set so you can re-run it after every change. The spreadsheet method is enough to start.

    Safety and privacy. Read the prompt injection note and, if you handle personal data in India, the DPDP note.

    Launch small

    Release to a limited group first, watch real use for a week or two, and fix what you find. A narrow launch with good monitoring beats a wide one you can’t see into. If you’d like a second pair of eyes on a feature before it ships, that’s part of what we do in our projects.

    Sources

    Want help putting this into practice?

    Book a free consultation and tell us about your team or school.

    Book a free consultation