Guide · Published November 14, 2026 · Reviewed November 14, 2026 · Ingenious AI team · Not legal advice
Token Human Review: Why Rubber-Stamping the Tool's Output Fails Every State's Test
The short answer: California's checker data names the failure directly: token review — a person rubber-stamping the tool's output — does not meet the human-decision test. Colorado's rules data reaches the same place from the review side: a reviewer who only restates the tool output does not meet the human-review duty. Token review is dangerous precisely because it looks like compliance — a human touched every file — while performing none of it. It arises from workflow design (throughput, defaults, deference), not bad faith, which means it is detectable with arithmetic and fixable with design.
Check your duties and dates — free Build your first notice — free
What is token review? It is a human step in an automated decision workflow where the person does not genuinely evaluate the case: they do not interpret the output critically, do not weigh other information, and in practice do not change outcomes. The legal consequence in the kit's data runs both ways — the decision remains automated for duty purposes (California), and the review offered afterwards is not meaningful (Colorado).
How token review arises: defaults, throughput, and deference
Three design forces produce token review in honest organisations. Defaults: the interface presents the tool's recommendation as the decision, with approval one click and disagreement a form — every interaction taxes dissent. Throughput: targets set per hour make genuine evaluation of each file arithmetically impossible. Deference: reviewers reasonably assume the tool knows more than they do, especially when they were never trained to interpret its output — California's first condition failing quietly.
Notice that none of these is a staffing failure. The same people, in a workflow with neutral defaults, feasible volumes, and interpretation training, review meaningfully. That is why the remedy in this guide is design, and why an organisation that discovers token review should fix the workflow before it blames the reviewers.
Detection one: the override rate and its distribution
A healthy review step changes some outcomes. The exact rate varies by tool quality and case mix, but a sustained zero or near-zero override rate across hundreds of cases is evidence the step is not evaluating — real tools err, real data has defects, real cases have context the tool missed. The distribution matters too: overrides clustered in one reviewer while others sit at zero suggests differing practice, not differing case mix.
Measure honestly: overrides divided by reviews, per reviewer, per month, from the queue's own records. The kit's review queue makes this a reporting exercise rather than an investigation, because requests, reviewers, and outcomes are already logged per case. If your current tooling cannot produce the rate, that absence is itself the first finding.
Worked example: Harbor Point Auto Lending
Example — fictional business, real duty structure. Harbor Point is a fictional lender that audited the human step in its California underwriting workflow — the step it had relied on as making its decisions human. The audit findings:
- Override rate: 3 overrides in 1,900 reviews (0.16%) over six months — statistically indistinguishable from no review at all.
- Time per case: median 70 seconds per review, against an underwriting file averaging 14 documents.
- Work notes: a sample of 30 notes finds 27 restating the tool's reason code verbatim, none referencing applicant-submitted information.
- Reclassification: the workflow is reclassified as ADMT (the carve-out fails); the risk assessment, pre-use notice, and opt-out routes are built — the work the token step had been assumed to replace.
- Redesigned step: neutral defaults, trained reviewers, feasible volumes; in the first quarter after redesign the override rate is 6.8% with work notes citing case facts in 29 of 30 sampled.
Detection two: time per case, and the shape of the work notes
Divide reviewer hours by cases closed. If the quotient is ninety seconds for decisions that took the tool a dossier to make, the review cannot have included reading the dossier. Time is not conclusive — some cases are genuinely clear — but as a distribution it is unforgiving: a median under a few minutes across a mixed caseload describes confirmation, not review.
Work notes tell the same story qualitatively. Notes that repeat the tool's reason codes ('score below threshold') without referencing the person's facts, submissions, or corrected data are restatements — Colorado's failure in the reviewer's own words. Sample ten notes a month and read them cold: could this note have been written without opening the file? If yes, it probably was. The positive standard is set out in Meaningful Human Review.
Why it matters legally, in the kit's own terms
In California, token review collapses the human-decision carve-out: the business that believed its decisions were human (and therefore outside ADMT) discovers they were automated all along — with the risk assessment, notice, and opt-out duties unbuilt. The classification error is retroactive in effect: every past decision was an ADMT decision made without the ADMT workflow. That is the most expensive version of the mistake, and it is entirely made of workflow design.
In Colorado, token review empties the review duty after the fact: the person was offered review, a person responded, and nothing was reconsidered. The record — which the business itself keeps for three years — documents the pattern. Connecticut's framing adds a third angle: since July 1, 2026 its profiling rights do not turn on the word 'solely', so a nominal human in the loop is not a structural answer there either.
Fixing it: redesign the step, then prove it
The fixes mirror the causes. Make the default neutral: the reviewer's options presented evenly, disagreement no more expensive than agreement. Set volumes from the work: pilot the true time a careful review takes and staff to it. Train interpretation: reviewers who can explain the output can argue with it. Require the work note that references the person's facts. And separate the reviewer's incentives from the original team's, so overturning is not a social cost.
Then prove the fix with the same detections that found the problem: override rates off zero and sensibly distributed, time per case in a plausible band, work notes that cite case facts. Run the classification again in the free checker — answering the role question as the workflow now actually operates — and file the before/after with the register. Token review is a design flaw; a redesigned, measured review step is the evidence it is fixed.
- Measure the override rate per reviewer per month — a sustained near-zero rate is token review until proven otherwise
- Compute time per case from hours and volumes, and compare it honestly with the work a real review requires
- Sample work notes monthly: restated reason codes without case facts are the failure in writing
- If review is token, reclassify the decisions as automated and build the duties the carve-out was assumed to replace
- Redesign defaults, volumes, and training — then re-measure and file the before/after evidence
Sources, as carried in the kit's own data: the duty summaries in app/rules.py and the checker data in app/checker_data.py, verified against primary or authoritative sources on October 28, 2026. Run the free duties checker for your jurisdiction, decision type, and the tool's role, and compare the kit's plans for the record, builder, and review queue. More from Ingenious AI is on the hub.
Related guides
- Meaningful Human Review: What 'Meaningful' Requires in Practice
- Colorado AI Act Automated Decision Notices: What to Send
- Adverse-Outcome Explanations in Plain Language
- ADMT Deadline Tracker: Which State Duties Land When
See all ADMT guides for the full set.
Frequently asked questions
What is token review?
A human step that rubber-stamps the tool's output without genuinely evaluating the case. California's checker data states it directly: token review does not make decision-making human.
What override rate proves review is meaningful?
No fixed rate is stated in the kit's data. The diagnostic is directional: a sustained zero or near-zero rate across many cases is evidence of token review; a plausible, distributed rate with case-specific work notes is evidence of real review.
What is the legal risk of token review in California?
The human-decision carve-out fails, so the decisions were ADMT decisions all along — made without the risk assessment, pre-use notice, and opt-out workflow the duties require.
Does Colorado treat rubber-stamping the same way?
In substance, yes: its rules data states that a reviewer who only restates the tool output does not meet the human-review duty.
Can token review be fixed without replacing staff?
Usually, yes — it is a workflow product. Neutral defaults, feasible volumes, interpretation training, and work notes that must cite case facts change the behaviour of the same reviewers, and re-measurement proves it.