An auditor does not care that your system is clever. They have no view on your architecture and no interest in your model choice. They care about three plain questions, and they ask them of every system, in every review.
The three questions
What did it touch? Which records, whose data, which systems. Not in principle — in this specific instance, on this date.
What changed? What state was different afterwards, and can you show the before and the after.
What did a human review? Who looked at it, when, and what were they actually checking.
That is the whole examination. Everything else is a variation on those three.
Why fast-built software cannot answer them
Because answering requires a decision made long before anyone asked. Something made a decision, nobody recorded why, and the trail stops at a log line that says the request succeeded.
This is not a gap you paper over later. It is an architecture decision you made on day one without noticing you were making it. Retrofitting an audit trail means going back through every write path and asking what should have been captured — which is most of a rebuild, done under time pressure, usually while someone waits for an answer.
AI makes the question sharper, not different
The three questions predate AI. What AI changes is how often a decision gets made without a person in the loop, and how hard it becomes to reconstruct the reasoning afterwards.
A model that classifies a transaction, drafts a customer reply or scores an application has made a decision your business is accountable for. "The model decided" is not an answer. Neither is a prompt log, unless it captures the input, the version of the model, the output, and what happened next.
What good looks like
Boring, and it works. Record activity like a black box: append-only, timestamped, tied to an actor — human or system. Keep the evidence somewhere that is not the same system that generated it. Make the answer a document you can hand over, not a scramble across three dashboards.
None of this is technically hard. It is just unglamorous, invisible in a demo, and the first thing cut when the deadline moves. That is precisely why it is missing from so much software that otherwise works well.
The practical test
Ask your team to produce, for one real transaction from last month: what the system touched, what changed, and who reviewed it. Give them an hour.
If it arrives as a document, you are in good shape. If it arrives as an explanation of how they would find out, you do not have a clever-software problem. You have an evidence problem — and it is much easier to fix before someone external asks. For FCA-regulated and compliance-heavy firms we design that evidence in from the first commit.
This is the part we do — the crossing from a demo to a system that survives production.