Applied labs: evaluation, cost, and guardrails

You counter the pathology by requiring evidence that was made by doing. At OmniCorp Financial, Priyanka Venn wants cost evidence she can defend. At OmniCorp Health, Nadia Haddad wants safety proof that survives a clinical review. At OmniCorp Logistics, Yusuf Demir wants workflow controls that hold during disruption. At OmniCorp Retail, Rafael Duarte wants a launch that will not leak customer data through a helpful assistant.

The lab standard is intentionally physical. Someone opens the case list, runs the workflow, captures the score, forces the attack, reads the log, and records the cost. Governance becomes a sequence of observable acts. That sequence is what lets a sponsor clearly distinguish an operating team from a team that has memorised the operating model. The distinction matters when failure arrives at speed.

Definition

The Evaluation Harness Protocol is the practice of building and running a repeatable test harness for an AI workflow before production. It includes golden cases, adversarial cases, acceptance thresholds, a scored run, and recorded residual failures. The protocol is not a discussion of quality: it is a dated run that shows what the workflow did against known cases.

A harness is small enough to run by hand and structured enough to run again. It turns quality from opinion into evidence. You do not need a perfect platform to begin. You need cases, expected behaviours, scoring rules, thresholds, and a place to write what still failed after the team improved the workflow.

When to Use It

Use the protocol before a pilot becomes production, before a prompt change ships, before a model or retrieval source changes, and before a vendor result is accepted as proof. Use it when the workflow affects customers, employees, clinical communication, financial judgement, logistics decisions, or regulated records. Use it whenever confidence is currently based on a demo.

Do not wait for an automated evaluation platform. Automation helps later. The first discipline is manual repeatability. A spreadsheet with case identifiers, inputs, expected outcomes, scores, reviewer notes, and residual failures is enough to start. A team that cannot run ten cases by hand is not ready to automate a hundred.

How to Apply It

1. 1Select golden cases that represent normal, high-value work the AI workflow must handle reliably.
2. 2Select adversarial cases that stress known risks: ambiguity, missing context, conflicting policy, unsafe instruction, and sensitive data.
3. 3Define acceptance thresholds before the run, including minimum score, blocked release conditions, and reviewer authority.
4. 4Run the cases against the actual workflow, with prompt, model, retrieval, and tool versions recorded.
5. 5Score each case, record residual failures, assign owners, and decide whether the release proceeds, changes, or stops.

Write the threshold before the result tempts you. For example: release requires 90 percent pass on golden cases, zero critical failures on adversarial cases, and no unsupported claim in a regulated output. A threshold written after the run is negotiation. A threshold written before the run is governance.

Worked example 1 of 3

OmniCorp Financial built a lending-summary assistant for complex consumer files. Bea Karlsson, model risk lead, asked Martin Hollis for twenty golden cases and ten adversarial cases. The golden cases covered standard income, identity, and collateral summaries. The adversarial cases included contradictory documents, a missing adverse-action notice, and a borrower note that looked relevant but was prohibited from the decision record.

MHMartin Hollis

The demo already looked accurate to the lending team.

BKBea Karlsson

A demo is not a run. Show me the cases it passed, the cases it failed, and the threshold it had to clear.

PVPriyanka Venn

Then show me the review hours, because the harness is now part of the operating cost.

Why This Works

The protocol works because it makes quality falsifiable. A stakeholder can argue with a slide, but a scored case forces the conversation onto evidence. The team sees which failures are rare, which are systemic, and which block release. Confidence becomes a result of repeated exposure to known risks, not a mood created by a polished walkthrough.

It also creates an operating memory. When a later prompt change breaks source citation, the team can compare the new run with the old one. When a regulator asks why the workflow was released, the evidence is already dated. The harness is not bureaucracy: it is the record that the team looked before it shipped.

Definition

The Guardrail Bench is a hands-on attack lab for an AI workflow. The team deliberately attempts prompt injection, data exfiltration, and over-permissioned tool calls, then proves each guardrail holds before production. The bench is not a policy review: it is a controlled attempt to make the workflow break its own rules.

A bench has named attacks, expected blocks, observed results, and corrections. It uses the real workflow path where possible, including retrieval, tools, identity, logging, and human approval. If the production workflow can call a tool, the bench tries to misuse that tool. If the workflow can retrieve documents, the bench tries to poison or overreach retrieval.

When to Use It

Use the bench before production, after any new tool permission, after a retrieval source changes, after a prompt policy changes, and after any incident or near miss. Use it when a workflow can see sensitive data, act through tools, influence customers, or draft regulated communication. Use it most when the team says the guardrails are obvious.

Do not reserve the bench for security specialists alone. Security should shape the attacks, but operators must run them too. A control that only the security team understands will fail during daily operation. The practitioner who owns the workflow needs muscle memory: what the blocked attack looks like, where the log appears, and who decides the release.

How to Apply It

1. 1List the workflow's guardrails: instruction hierarchy, data boundaries, retrieval filters, tool permissions, human approvals, and logging.
2. 2Create attack scripts for prompt injection, attempted data exfiltration, over-permissioned tool calls, and bypass of human approval.
3. 3Run each attack in the controlled environment using the same identity and connectors expected in production.
4. 4Record whether the guardrail blocked, degraded safely, alerted, logged, or failed, with exact evidence captured.
5. 5Fix failures, rerun the same attack, and price the review, monitoring, and incident-response hours into the operating model.

The bench is successful when it produces a boring answer under hostile input. The model refuses the injected instruction. The retrieval layer withholds the document. The tool call stops at the approval gate. The log shows the attempt. The owner can explain the evidence in two minutes. Anything less is a rehearsal that found work still to do.

Worked example 1 of 3

OmniCorp Logistics ran the bench for its disruption assistant. Anders Nilsen created attacks against a prompt that asked the assistant to ignore dispatch policy, a retrieval attempt for customer payment notes, and a tool call that tried to reassign a truck without supervisor approval. Claire Beaumont required the bench to use a real supervisor account in a controlled environment, because paper permissions had already fooled the team once.

Why This Works

The bench works because it removes comfortable ambiguity. A control either holds against the attack or it does not. The team sees the failure path before an attacker, an employee, or a malformed document finds it in production. That experience changes behaviour. People stop saying the policy exists and start knowing where the block occurs.

It also prices safety honestly. Guardrails cost time: attack design, reruns, monitoring, approvals, and incident response. When Priyanka Venn saw the bench results, she accepted the additional review hours because they prevented a larger cost: an emergency shutdown, two weeks of rework, and a regulator's letter asking why tool permission was never tested.

OmniCorp Logistics ran the labs for the disruption assistant before winter peak. Yusuf Demir began with a strong demo and a real operational need. Claire Beaumont converted the demo into a harness. She chose historical disruption cases as golden cases, added adversarial cases for missing depot capacity and conflicting route rules, and required a threshold before the first run.

The first harness run passed most normal cases and failed one critical adversarial case: the assistant recommended a route using a closed depot. Anders Nilsen recorded the residual failure, tied it to retrieval freshness, and assigned the fix. The rerun used the same case and passed. Priyanka Venn added the review and rerun hours to the workload cost rather than hiding them.

Then the team moved to the Guardrail Bench. Anders attacked the workflow with prompt injection, payment-note exfiltration, and an over-permissioned truck reassignment. The prompt injection was refused. Payment notes stayed outside retrieval. The truck reassignment opened only as a draft for Claire's approval. One approval-bypass attempt failed closed but lacked a clear alert, so the release waited until logging was repaired.

The final release record contained the harness score, residual failures, bench attacks, guardrail evidence, control labour, and owners. Rosalind Achebe saw an operating case, not slideware. The assistant launched with narrower confidence and stronger proof. The team moved faster because it had already done the work that a live incident would otherwise demand.