Two failure stories.
A SaaS product ships an AI search feature. It demos well. The demo is run against fresh data, curated for the presentation. Three weeks after launch it is returning confident, well-formed wrong answers, because the underlying database is updated on a schedule the model knows nothing about. The feature reads a snapshot from last Tuesday. Customers are making decisions based on information that has not been current for a week. Customer support eventually surfaces this. Engineering traces it to the data path. Nobody had documented which system the feature was reading from, at what cadence, or who owned the refresh.
A second product ships an AI summary feature. In QA it works. In beta it works. Six weeks after launch someone in the product team notices the customer-facing summaries have shifted in tone. Not drastically. Enough. The support team has been fielding an uptick in confused customers but nobody connected it to the AI feature until the product manager pulled the timeline. The model had not changed. A prompt adjustment made in the first week of launch had drifted the output, and there was no audit trail and no rollback. Nobody knew which version of the prompt was running in production.
Both are integration failures. In each case the model performed as it was supposed to. The question neither team answered before starting was not which model to use but what the feature needed from the rest of the system to behave correctly in production.
That is the question this article works through.
What "adding AI" actually requires operationally
When a product manager or a CTO describes adding an AI feature to an existing SaaS product, the picture is usually: pick a model, call the API, ship. That is the shortest description of the work. The things that decide whether the feature is still working correctly a month after launch are around that API call, not inside it.
Six items. They are not unique to AI. They fail specifically when AI is added because language model output varies per request, failure is often quiet rather than loud, and the cost of a bad prompt version running for a week does different damage than the cost of a bad code version running for a week.
A documented data contract from the feature to your live product state. Not a daily export with an undefined refresh schedule. Not a cached snapshot from a nightly job. A contract that names the source system, the fields, the refresh rate, and who owns it. When that contract breaks or silently degrades, the feature breaks in ways that look like model failure but trace back to the data path.
Per-request cost and latency instrumentation, visible in the dashboard the team already reads. Not a new monitoring tool. The one they already look at every morning. If this feature doubles your API cost under sustained load, that signal belongs next to the rest of your product health indicators, where it will actually be seen.
A rollback path for when the feature ships worse output than the previous version. This requires treating your prompts and system instructions as versioned artefacts rather than configuration files. Without it, reverting to a known-good state means a code deployment, manual log review, and guesswork about which change caused the regression.
A review queue for output the model is uncertain about, with a named human owner. Not every feature needs this. Any feature that produces text that a customer reads, or drives a decision that affects a customer account, probably does. The queue does not need to be a complex system. It does need to exist before the feature ships.
A versioning discipline for prompts and system instructions. Separate from the rollback point above, because the reason here is traceability, not recovery. A prompt stored in a comment, a private config file, or a document that only one person knows about is a prompt that changed and left no trail.
A plan for the change in your support workflow. Customers will ask about the AI feature and they will ask about its mistakes. A support team that cannot explain why the AI produced a specific output is a friction point. That friction compounds. It belongs in the scoping conversation, not in a post-launch incident review.
Three shapes, three integration burdens
An AI feature in a SaaS product typically takes one of three forms. The integration burden differs by form. Knowing which shape your feature takes before scoping the work changes the questions you have to answer.
Retrieval. The model reads from your product data and answers a question grounded in it. Data contract, freshness guarantees, permission model, and a filter path that respects what each user is allowed to see. The first failure story in the opening is a retrieval integration failure. It is also the most common shape for a first AI feature, which may be why this particular failure pattern is so frequent.
Generation. The model produces text, code, or an artefact that goes back into the product surface. Quality gate before output reaches a customer. Audit trail of what was produced and when, tied to the prompt version that produced it. Rollback path for a bad prompt version. The second failure story above is a generation failure.
Decision assist. The model recommends or ranks something, either for a user to act on or for an automation to act on directly. This is the shape where the failure surface is widest and most often underestimated at scoping time. Confidence handling, a definition of what a wrong recommendation looks like, a deterministic fallback path when the model refuses or returns output outside expected bounds. A system that passes a confident recommendation directly into an automated workflow without a fallback is running without a safety valve. What that failure surface looks like in a production system, and how it tends to be discovered.
Most product teams can identify which of the three shapes their feature falls into before writing a line of code. That identification changes what you need to answer before you start.
Five questions before you scope this work
Teams that have done AI integration before tend to arrive at kick-off with these questions already answered, or at least drafted. Teams doing it for the first time tend not to. The cost of skipping them appears in the third week, not the third year.
- Which specific customer job is this feature doing, and how will we know it did it. Not "improve experience" or "reduce manual work." A testable job description. If you cannot state the success condition for a single request in one sentence, the feature is not scoped yet, and the engineering brief will carry that ambiguity all the way to production.
- What is the data path from the customer's live product state to the model, and back. Named systems, named contracts, named owner. This question surfaces more deferred architectural decisions than any of the others. What a structured data interface for this path looks like is what you are specifying here, before the first line of integration code is written. For a team that needs this layer designed before committing to a build, klim.expert starts here.
- When the model returns something we did not expect, what does the feature do next. Show the code path. Not "we will handle it." The branch, the fallback, the message to the user, the review queue trigger if applicable. If this answer is not ready, the error path has not been designed.
- Where in our existing monitoring will the cost and latency of this feature show up. Not a new tool. The one the team already reads. If the answer is "we will add monitoring after launch," cost visibility will not exist until the first unexplained bill, which is a worse moment to find out.
- When the vendor changes the model version or deprecates a provider, what happens to our feature. Named person, named playbook. A feature with no answer to this question has a dependency that nobody owns, and nobody will own it until the dependency breaks.
Teams that work through these before writing a job spec or an engineering brief ship cleaner integrations the first time. Not because the questions are subtle but because they surface assumptions that would otherwise be discovered in production.
The scoping decision
The five questions point to a specific slice of work: architecture decisions before the build, data contract design, instrumentation plan, rollback path. That work comes before prompt engineering and before the API integration work. It is also what determines whether the feature is readable and maintainable six months later, by whoever is running the product then, including people who were not in the room when it was built.
A team that has worked through these five questions knows what they are building and can brief an engineer or an external development team cleanly. A team that has not is about to discover the questions in production, one at a time.
If you are at the scoping stage and want a working integration rather than an open requisition: one senior engineer, a working system in roughly a week, documented so any future developer on your team can service it. No proprietary platform, no monthly fees to a tool you do not control, cost agreed before the work starts. klim.expert
