Checklist · 6 min

The 20-point audit we run on AI-generated code

Twenty checks, in order of what actually leaks customer data. Most of them a competent developer can run — the ones nobody can answer are the point.

This is the checklist we work through on AI-generated code. It is deliberately plain: a competent developer can run most of it, and the items nobody can answer are usually the ones that matter.

Access and data boundaries

1. Can an unauthenticated request read data? Call your main endpoint with no token and see what comes back.

2. Can one customer read another's records? Test with two accounts, not one.

3. Are row-level security rules written for every table, not just the obvious ones?

4. Do admin routes check a role, or only hide the button in the interface?

5. Is anything sensitive returned by an endpoint that does not strictly need it?

Secrets

6. Does any key with write access reach the browser bundle?

7. Has anything ever been committed to git history, even if removed since?

8. Are production and development using separate credentials?

Input handling

9. Is every user input validated server-side, not only in the form?

10. Are database queries parameterised throughout — no string concatenation anywhere?

11. Are uploads restricted by type and size, and stored where they cannot execute?

12. Is model output treated as untrusted input when it flows into a query, a command or the page?

Failure behaviour

13. What happens when a third-party API times out — retry, fail loudly, or hang?

14. Are errors surfaced, or swallowed by a catch block that logs nothing?

15. Is there a rate limit on anything expensive to call?

16. If the model returns nonsense, does a human see it before a customer does?

Evidence

17. Can you show what the system touched for one specific transaction last month?

18. Is there an append-only record of who changed what, and when?

19. Are backups taken, and has a restore actually been tested?

20. Could a person outside your team follow the code to understand what it does?

How to read your score

This is not pass or fail. Most applications built quickly fail several, and that is the expected result rather than an indictment — these are the items that are invisible until someone looks.

What matters is which ones you fail. Items 1 to 5 are the ones that leak customer data, and they are worth fixing this week. Items 17 to 20 are the ones that turn a manageable incident into an unbounded one.

If working through it raises more questions than answers, a Reality Check runs the whole list against your actual system and prices the fixes.

This is the part we do — the crossing from a demo to a system that survives production.