Incident recognition and escalation

Someone spots a fabricated figure in a draft before it goes out. They correct it. It took ninety seconds. They do not report it, because nothing happened: the control worked, the output was fine, and reporting it would mean a form, a conversation, and the implication that their team's AI use is a problem. Multiply that by every competent person in the organisation.

The counter is to make the near miss the primary reportable event rather than a lesser category of one.

Definition

Four steps in fixed order. Recognise: name what class of failure this is, before deciding whether it matters. Contain: stop the flow, which means stopping further outputs of the same class, not merely correcting this one. Preserve: capture the prompt, the output, the timestamp, the account and the version, before anything is regenerated or cleared. Escalate: tell the named owner and the reporting path, including when you have already fixed it.

The order is not a suggestion. Preserve before escalate, because the first thing escalation triggers is somebody trying to reproduce the problem, and reproduction destroys the evidence of what actually happened. Contain before preserve, because containment is time-critical and preservation is not.

When to Use It

Use it for every near miss, not only for events with consequences. A fabricated citation caught in review is the same event as one that reached a customer, differing only in whether a person happened to be looking. Treat them identically and the near miss becomes your early warning system.

Use it also when the failure was your own: your prompt, your data, your oversight. Self-reported near misses are the highest-value entries in any log, and an organisation that punishes them will not receive a second one.

How to Apply It

1. 1Name the failure class before assessing severity: fabrication, disclosure, bias, drift, misuse, or unavailability.
2. 2Contain the class rather than the instance, and accept that this may mean pausing a workflow that is mostly working.
3. 3Preserve prompt, output, timestamp, account, tool version and who was in the loop, before anyone regenerates anything.
4. 4Escalate within a stated window even when the issue is already resolved, and record it as a near miss if no harm reached anyone.

Worked example 1 of 3

A drafting assistant at OmniCorp Financial produced a customer letter citing a regulatory reference that does not exist. A servicing agent noticed, deleted the sentence, and sent the letter. Two weeks later a colleague found the same fabricated reference in a different letter. Marisa Delgado walked the four steps with them.

MDMarisa Delgado

Recognise. What class is this?

SAServicing agent

Fabrication. It invented a rule reference.

MDMarisa Delgado

Contain. What stops the class, not the sentence?

SAServicing agent

Pausing any letter that cites a regulatory reference until we check them all.

MDMarisa Delgado

Do that now. Preserve. Have you still got the original output from a fortnight ago?

SAServicing agent

No. I edited it in place.

MDMarisa Delgado

Then we have lost the first instance, and that is the cost of the quiet fix. Escalate anyway, and log it as a near miss with what you remember.

The review found the fabricated reference in nine sent letters over five weeks. Every one had passed a human reader. Nobody had reported the first.

Why This Works

Naming the class before assessing severity is what makes containment possible. Severity-first thinking evaluates the instance, concludes correctly that this instance was minor, and stops. Class-first thinking asks what else has this shape, which is the only question that finds the other eight letters.

Preserving before escalating works because AI failures are frequently unreproducible. Regenerate the same prompt and you may get a clean answer, at which point the incident is quietly reclassified as a misunderstanding and the underlying pattern survives. The artifact is the only durable evidence that it happened at all.

Dr. Naomi Ellery was told, informally and in a corridor, that a clinical summarisation tool had twice omitted an allergy from a discharge summary. Both had been caught at signature. Neither had been reported.

Recognise: not fabrication but omission, a distinct class, and one that no evidence check for invented content would ever catch. Contain: she suspended the summarisation of any note containing an allergy field, which affected roughly a third of throughput and generated immediate pushback about the burden that placed on clinicians. She held it, because the class was omission of safety-critical fields and the containment had to match the class rather than the two known instances.

Preserve: the two known cases had been edited in place and were unrecoverable, so she preserved what existed, the signed versions and the underlying notes, and required raw capture from that point on. Escalate: to the clinical governance lead within the day, and to the supplier, whose response was that the model was not designed to guarantee field completeness. That sentence was the finding. The tool had never promised what everyone had assumed it did.

The retrospective review across four months found eleven further omissions, none reported, all caught at signature by clinicians who had assumed they were the only one. Eleven separate acts of competence, each individually correct, together forming a blind spot that had lasted a third of a year.

Four steps and an honest near-miss log will let a team see its own failure patterns before they become incidents. They will not tell you when a regulator must be notified and within what window, how to preserve evidence without compromising privilege, or how to run a post-incident review that produces a change rather than a document. Building incident management that stands up to external scrutiny is the capability the paid programs develop next.