How We Ship an AI Feature in One Week

What "shipping in a week" actually means

This is a walkthrough of the method as it played out on knowster.chat, our own product. We use it because the arc is fully reconstructible from the artifacts and because the same shape applies to customer engagements. Where a section describes something that was true for knowster.chat, we say so plainly. Where a section describes what the same step looks like on a customer engagement, we say so plainly. Days are numbered from zero to five to make the arc readable; some decisions actually landed a day earlier or later, and we have compressed a few for clarity.

At the end of week zero, knowster.chat did not exist as a running system. There was a job to do: site visitors were reading pages and leaving without a way to ask a specific question that content could answer. No product was doing that job yet. At the end of week one, a running assistant was live on the site, documented, and acceptance-tested against the job. On a customer engagement, the before-state is a manual process eating a person's time and the after-state is the same running system on their infrastructure, tested on their data.

"In a week" is not a demo. A demo runs on prepared data in a controlled environment and shows well precisely because the rough edges have been removed. It does not represent what the system does when a real visitor asks a question the prepared data does not cover. A demo is not what we mean.

"In a week" is also not everything a customer might want. The first working version does one named job. It does not cover every edge case, every integration, or every output surface the customer might eventually need. Those come later. The acceptance test is the scope, and the scope is decided before the build starts, not after.

Day 0 to Day 1

The scoping conversation is not a requirements document. It runs to three questions, and all three need a yes before "in a week" is an honest answer for a particular engagement.

For knowster.chat, we asked these three questions of ourselves.

First: do we have the data path and the access we need on the first day? Not "we can probably arrange it." On day one. If the data path is still unclear when build starts, the first two days become discovery, not construction, and the timeline collapses.

Second: can we name the one job in one sentence? For knowster.chat, the job was: the assistant answers a visitor's question about this business from content the business has already published on its site. One sentence. When a customer wants three jobs in the same sentence, the scope is not a week.

Third: is there a named person on our side who can run acceptance in five days? Not "someone will be available." A named person with the authority to look at the running system on day five and say yes or no. Without that person, the acceptance test has no authority and "shipped" means nothing. On knowster.chat, that person was us: the same team that built the system ran the acceptance test. On a customer engagement, it is a named person on the customer side with that same authority.

The cost of "someone will be available" is worth naming. When the answer is not a specific name but a general assurance, what follows is: the build completes, the acceptance test is ready to run, and nobody is available to run it. The engagement enters a waiting period whose length is set by the customer's calendar, not by the delivery team. A calendar week passes without a yes-or-no verdict, and every deliverable is now retrospectively unclaimed. The fix is to set a named acceptance date before build starts, the same way the scope is written before build starts.

The first artifact out of day zero is a short written scope: the one-sentence job, the three yes answers, and the named person responsible for acceptance. That document is the reference the whole week runs against. If something is not in it, it is not in this engagement.

The same conversation looks like this from the consulting seat: the discovery call for an AI engagement.

Day 1 to Day 2

Four decisions get made in the first two days, and each gets made once. The reason is not process discipline for its own sake. A decision made mid-week is made against existing code rather than before the code exists. That puts pressure on the answer: the expedient choice looks more attractive when the alternative is rewriting two days of work.

Model choice

For knowster.chat, the job is answering factual questions about a business from a structured knowledge base. The choice was a managed LLM API suited to retrieval-augmented generation: reliable at following the knowledge base contents, low hallucination rate on this particular task shape. We do not name the provider because the choice is specific to this job and this data volume. A different job, or a significantly different data structure, would change the answer.

For the patterns behind plugging a custom LLM into a live product, the piece on custom LLM integration in practice covers the technical decisions in detail.

Where the model runs

Everything runs on our own infrastructure: one VPS under a managed hosting stack. Application and data are mounted separately. A deploy replaces the code and cannot touch the data. That is a deliberate early decision, not a consequence of something else. Every deploy and every rollback since has relied on it.

For the way this infrastructure separation plays out when adding an AI feature to an existing product, the piece on the shape of adding one AI feature to an existing product covers the same structural decision in a different context.

On a customer engagement, the same decision means the customer runs the system on their infrastructure with no platform fee once the engagement ends.

Data path

The knowledge base for each site is a discrete artifact: a structured file built by an ingest job that runs from a scan queue, asynchronously from the live chat queries. Chat reads from the knowledge base. Chat does not write to it. This separation is decided on day one, not because it is clever architecture, but because it is the only data path that does not create a write conflict between the build job and the live queries. A choice that looks structural is actually an operational constraint written in early.

Failure mode

What does the system do when it cannot give a useful answer? The ingest job can return no content if the site has nothing readable. A scan that finds nothing fails with a reason rather than finishing with a green status that misrepresents what was built. Certain link types are refused by name to prevent the assistant from answering about a platform rather than the business's own content. The chat endpoint has a defined fallback response. These are not edge cases added in week two. They are part of the specification, written alongside the model choice, because "what does the system do when it breaks" is a day-one question, not a polish step.

Day 3 to Day 4

Once the four decisions are made, the build is short. That is the point. If it stretches past two days, the decisions in day one and two were not actually made: something was deferred and is now being decided mid-build, which is the slower and more expensive place to decide it. Go back, not forward.

The knowster.chat build produced three discrete artifacts: the knowledge base per site, the chat endpoint, and the widget script. Each has a clear boundary. The knowledge base is built offline by an ingest job from the scan queue. The chat endpoint reads from it at query time. The widget loads on the site and talks to the endpoint. At runtime, none of the three reads or writes to another's data.

Two artifacts go in alongside the code from the first day of build, not added afterward.

The first is cost and latency instrumentation. In the knowster.chat build, chat events and chat requests were initially tracked as a single number. Chat events are widget loads and automated interactions, including bot traffic that the widget sees but that represents no real visitor intent. Chat requests are real messages sent by real visitors. They are two different things with two different denominators: one is a measure of deployment reach, the other is a measure of product use. When they are combined, the dashboard shows a healthy composite number that overstates actual conversation volume. A period of that combined metric, before a correction was made, meant that operational decisions about whether the assistant was being used were made against a number that counted bot pings alongside genuine questions. Splitting the counter into two named metrics required rebuilding the panel and re-baselining the threshold that triggered a low-activity alert. The cost was low, but the correct baseline had to be re-established from scratch. Build the measurement with the thing being measured, because a metric added after the fact has to explain the delta from the metric before.

The second is a running architecture document. It gets written as the build proceeds so that by the time the acceptance test runs, the reader can review a document that describes what they are accepting before they accept it.

For the operational visibility argument behind building instrumentation in from the first day of build, the same reasoning is at work in the piece on back-office automation with AI.

Day 4 to Day 5

The handoff is what the end of the engagement delivers, and it is more than a repository link.

On knowster.chat, the engagement did not hand off to an external team: we owned the system through build and into production. But the handoff artifacts were built in from the first day of the build regardless, because a build that does not produce them is not a build that runs reliably in production under any team's ownership.

What those artifacts were on knowster.chat: the full code history in a versioned repository, the architecture document written as the build proceeded, a cost metric tied to a dashboard that read real conversation volume, a playbook for the three or four most common changes a team will need to make (a new knowledge base build triggered by a site update, a revised prompt version, a new failure-mode branch for a new content type), and the deploy script with a rollback checkpoint from the first day of build. If an acceptance test fails, roll back to the last checkpoint, diagnose, fix, and run the test again. That mechanic exists from day one.

The playbook is not a manual for the whole system. It covers the changes a non-specialist operations team will need to make in the first six months. It names what to do, what not to touch, and who to call when something outside the playbook appears. That boundary is part of the handoff.

The acceptance test on knowster.chat covered: a visitor question answered from the knowledge base and not from the model's general training, a new site scan completing and producing a usable knowledge base, and the defined failure path returning an error rather than a fabricated answer. That is a real run, not a walkthrough.

For what an LLM engineer's responsibilities look like during and after the handoff period, what an LLM engineer actually does covers the same phase from the hiring perspective. For the way ERP-side data sources affect handoff complexity, the piece on ERP automation with AI covers the same structural challenge in a different context.

On a customer engagement, this same package is what the customer receives, and the acceptance test runs in front of their named acceptance person on their data. A system built this way, with documented architecture, a cost dashboard, and a working rollback, is a system your team can take over and run without the people who built it.

What "in a week" leaves out, by intention

Three things the reader should not expect from a first working version.

No polished UX. What the user gets is a functional surface they can operate. It is not a designed product. Visual detail, spacing, and interaction refinement are later work. The acceptance test passes on the job, not on the finish.

No edge cases beyond the named job. The acceptance test is the scope. For knowster.chat, the test covers the main path and the defined failure path. It does not cover every content type a site might contain, every browser a visitor might use, or every way a visitor might phrase a question outside the intended job. Those are real problems. They are second-engagement problems, not week-one problems.

Some of the traps the knowster.chat build paid for illustrate this directly. Preview chats from the dashboard leave no trace in the system, by design: any monitoring that tries to detect whether a test was run reads a false zero. A test written early resolved the data directory at module import time, which meant it pointed at the production database during test runs. A container restart does not reload environment variables from the compose file, and the documentation does not say that clearly. None of these blocked the acceptance test. All would have mattered if they had been treated as week-one scope rather than second-week observations.

No integrations not in the scope document. A second data source, a second output surface, a third-party notification feed: each is a separate engagement or a planned extension. A vendor who claims all of it in a week is either not delivering all of it, or "in a week" is the wrong phrase for what they are offering.

For an analysis of what happens when scope limits are treated as negotiable mid-build, the piece on when agents break covers the failure surface in detail.

When the pattern does not apply

The pattern behind knowster.chat's week does not fit every case. Three scenarios where "in a week" is the wrong answer for a customer engagement.

The scoping questions from day zero all answer no. The data path is unclear. The job cannot be named in one sentence. There is no named acceptance person. When this is the case, the first engagement is a scoping engagement, not a build engagement. Calling it "ship in a week" sets a frame the actual work cannot satisfy. The right answer is to name what the first engagement actually is.

The customer team is not resourced to accept a running system in five days. The acceptance person is named but not available during the build week. IT can grant access, but not in the first two days. The build is ready but the acceptance test cannot be run. This shifts the timeline, and the right response is to name the constraint and set a date that matches reality rather than the original intention. The decision between hiring a consultant and building in-house affects this resource and availability question directly.

The underlying data needs cleanup before any AI work is honest. The acceptance test runs against data. If that data contains duplicates, conflicting records, or missing fields the system depends on, the test cannot pass on the first run for reasons unrelated to the build quality. Every significant issue in the knowster.chat build traced back to a lie in the data: limits that promised more than they served, a monitor that reported OK while checking nothing, analytics that counted automated traffic as real conversations. When something looks green, the first question is whether it can go red. Data cleanup is a different engagement from AI feature build, and treating it as week-one prep for a week-one build is how a week becomes three weeks. The piece on AI automation for a small business covers the data-readiness question in the context of smaller operational teams where this gap appears most often.

Closing

Three things to check when a vendor claims "in a week."

A real named deliverable. The one-sentence job, in writing, before work starts, unchanged at acceptance. Not "you will have a working AI system."

A real handoff. Repository access, architecture document, cost instrumentation, playbook for the common changes. Not a demo and a Slack channel.

A real acceptance moment. The named person, the production environment, the customer's data, the run that either passes or does not.


This is what a week looks like when the goal is a working system, not a demo. That is what klim.expert is for.

enjoyed this? follow me!

X / Twitter LinkedIn GitHub

share this!

← Back to blog