Consumer Duty asks firms to deliver good outcomes for retail customers and — the part that becomes an engineering problem — to monitor whether they are, and evidence it.
Add an AI feature that touches customers and that monitoring obligation lands on a system that, by default, keeps almost nothing worth monitoring.
This is a description of the engineering consequence, not compliance advice. Your compliance officer owns the interpretation; the point here is what the interpretation requires the software to have kept.
Where the obligation meets the build
Monitoring outcomes means being able to say what happened to customers, in aggregate and individually, and to notice when a group is being served worse than others.
For a deterministic system that is straightforward — the rules are in the code, so the behaviour is inspectable. For an AI feature it is not, because the behaviour is not written down anywhere. The only way to know what it did is to have recorded what it did, at the time, in a form you can query later.
Which means the monitoring capability is a design decision made at build time. It cannot be added retrospectively, because the evidence for the past six months either exists or does not.
What "retrievable record" means in practice
For any AI interaction that touches a retail customer, being able to reconstruct it later means keeping, at minimum:
The input. What the customer actually asked or submitted, and the context the system added.
The output. What was returned. Not a summary — the text or decision as it was.
The version. Which model, which prompt, which configuration. Prompts change weekly in an active product; without the version, an output cannot be explained.
What happened next. Did a human review it? Did the customer act on it? Did they come back? The outcome is the part Consumer Duty is actually about, and it is the part almost nobody joins to the interaction.
The timestamp and the actor. When, and on whose behalf.
The vulnerable-customer problem
The Duty asks for particular care with customers in vulnerable circumstances. A language model has no concept of vulnerability unless something in the system gives it one.
Two engineering consequences follow. First, if your firm identifies vulnerability, that signal has to reach the AI feature — otherwise the feature is the one part of the business that ignores it. Second, your monitoring has to be able to segment: not "how did the assistant perform" but "how did it perform for this group compared with everyone else." That is a query you can only run if the segment is in the record.
What the monitoring itself needs
Recording is necessary and not sufficient. The obligation is to notice.
In practice that means a regular, automated look at the recorded interactions for the things that matter: rates of escalation to a human, cases where the customer immediately asked again, outcomes that differ by segment, and outputs that failed an automated quality check. It also means an alert when any of those move, rather than a report nobody reads.
Boring, unglamorous, and the difference between "we monitor outcomes" as a claim and as a fact.
The practical test
Pick one customer who used an AI-touched journey last month. Ask your team to produce, in writing: what the customer asked, what the system said, which version produced it, whether anyone reviewed it, and what happened afterwards.
If that takes an hour, you are in good shape. If it takes a week and ends in an apology, the gap is not a compliance gap yet — it is an engineering gap that becomes a compliance gap the first time someone external asks. We design that layer in from the first commit for FCA-regulated and compliance-heavy firms; if you would rather know where you stand first, a Reality Check is a week and a fixed fee.
This is the part we do — the crossing from a demo to a system that survives production.