Back-office automation projects usually fail for the same reasons that every integration project fails. The model is the last thing to blame. I have seen this pattern in ERP work with no AI involved at all: records disappear between the two systems and someone in accounts payable discovers it a week later from an unpaid supplier, a process that no longer matches how the business actually runs, and no way to verify the output is correct once the implementer leaves. Add AI on top and you add two more failure modes. Leave all five unaddressed and you will have a system that demonstrates well, costs more to run than expected, and still has a person checking every row because the output was never trusted.
This is not about choosing the right AI vendor. It is about what the project has to include before the model goes in.
Why back-office automation breaks before the AI arrives
Three patterns show up in every project that fails. None of them started with AI.
An invoice is raised in the ERP and never reaches the accounts system. Accounts payable finds out a week later, when the supplier calls about an overdue payment. When someone looks into it, the integration was built against documentation that was accurate when it was written. A required field turns up blank on some transaction types. A document category that never appeared during the build shows up on the third day. Neither team owns the reconciliation, so neither team fixes it.
A contractor automated a process and handed it over. The system worked on delivery. Six months later, a new document type arrives. The change is minor. Making it requires the person who built it, because there is no way to test on real data without their involvement. Every exception becomes a support ticket. The "autonomous" system still has a human in the middle of every non-standard case. Autonomy was promised, but the contract for how the system handles edge cases was never written.
The process is formally automated. A team member still opens the output each morning to check it. Not because their job description says to, but because an error appeared once and nobody confirmed the system was fixed. There is no test they can run on real data to verify the output is correct. The result is a shadow spreadsheet and a manual verification queue sitting alongside the automated process, which defeats the purpose of automating it.
All three are back-office automation failures. None of them needed AI to happen.
What AI adds on top
Add AI to those three unresolved and each one becomes harder to diagnose. The failures are less reproducible and the root cause is harder to attribute. Even with the three handled cleanly, AI introduces two failure modes that most integration work does not encounter.
The first is that the output is non-deterministic. The same invoice, processed twice, can produce outputs that differ in a way that passes every schema check. Structured output enforcement at the API level solves the format problem: you get valid JSON, correctly typed fields. It does not solve the meaning problem. The model categorises an operating expense under the wrong departmental budget. The JSON is correctly formed, the field type is correct, the record processes without an alert, and finance finds the discrepancy at month-end. Monitoring that catches a malformed response catches nothing here. The design decision is made before the first production call: where a wrong answer has low cost, log it and correct later; where it involves money or a compliance obligation, route it to a human review queue and do not process automatically. Concrete cases of this kind of semantic drift causing downstream failures are documented at klymentiev.com/blog/when-agents-break.
The second is that the operating cost is variable. Token consumption is not fixed: it depends on output length, which the model determines, and on context size, which grows with each step in a multi-step process. A prompt change, a new document type, or a heavier processing period can change monthly costs without triggering any alert in a standard monitoring stack. You cannot manage what you are not measuring. The instrumentation has to be in place before the first production run. The data-contract layer, where you control what flows into each call, is the right place to build it. How that contract layer is structured so it stays maintainable is discussed at klymentiev.com/blog/building-mcp-servers.
What a working system looks like in concrete terms
Working does not mean "runs without errors." It means you can verify the output is correct without a person checking every row.
A working back-office automation system has five properties. The interface between systems is documented, owned by a specific person, and tested against real production data before launch. The downstream error path is defined: when the AI returns something outside the expected range, there is an alert, a queue, and someone responsible for it. The operating cost per process (token consumption per call path) is measured and visible as a running figure, not a surprise at billing time. There is a test path on real data that someone who did not build the system can run to confirm it is processing correctly. And the system can be taken over by any competent developer six months later without the original builder.
That last point determines whether the first four hold after the engagement closes. The integration layer under the model is what makes the system maintainable. The model is the last piece, not the first. A model sitting on an undocumented or fragile integration produces failures that are hard to attribute and expensive to fix. The failure surface here is the same as in any custom LLM integration: the integration work decides whether the system is reliable, not the model selection.
What to ask before you hire anyone for this
Five questions. If the person you are talking to cannot answer each one specifically, the project will spend most of its budget on discovery that could have happened in the first meeting.
One: how do you establish the data contract. Not "we document the interface." Who confirms what the production data actually looks like, what happens when it does not match the documentation, and who owns the discrepancy when it appears in production.
Two: what happens when the output is wrong. Where does it surface, what does the alert look like, and does the wrong output flow downstream silently or get held for review.
Three: how do I run the test after you leave. A path I can run myself, on real data, in a staging mode on the real system, without calling you to set it up.
Four: what does operating cost look like if our volume doubles. What is instrumented today, where can I see cost by call path, and what does a prompt change do to the monthly figure.
Five: how is this built so it stays serviceable in a year, meaning a readable, documented architecture any developer can take over, with no proprietary platform needed to run it.
Those five questions map to the five properties above. The first three cover the classic integration failures. The fourth covers cost drift. The fifth determines whether the whole thing holds after the implementer is gone.
One senior engineer, your codebase, a working integration in about a week. Readable, documented architecture that any developer can service. No proprietary platform, no monthly platform fees, cost known before the work starts.
