Gatekeeper

Every form a field technician submits, text and photos, has to be reviewed by a human who approves it or returns it. Reviewers are the bottleneck. The obvious fix, letting an AI approve forms directly, is also the obviously wrong first move: nobody should hand production write access to a model on faith.

So the AI has to earn the job the way a new hire would: by being reviewed itself. In learning mode, Gatekeeper evaluates every form in parallel with the human reviewer. Its verdict, approve or return, is recorded and never applied. The human's decision remains the only one that counts.

The interesting part is what that produces: because every AI verdict is stored next to the human's real decision on the same form, the system builds its own evaluation dataset from live production, for free, forever. No hand-labeled test set that goes stale.

• Promotion is a hard gate, not a feeling: at least 95% agreement with human reviewers over at least 200 cases before write access is even discussed.
• Disagreements are the most valuable output: each one is either an AI error to fix or an inconsistency in the human criteria worth surfacing.

A daemon on my VPS, kept alive by systemd, pulls submitted forms from the OfficeTrack API around the clock, runs the review, and stores each verdict pair. The write path to production exists in the design and stays disabled until the gate is passed.

Mode

Learning mode: every AI verdict is recorded next to the human decision, and never applied.

Runtime

A daemon on my VPS, kept alive by systemd, reviewing forms 24/7 since June 2026.

Review scope

The full submitted form, text and photos.

Evaluation set

Built from live production, continuously: one AI verdict stored beside one real human decision, per form.

Promotion gate

At least 95% agreement with human reviewers over at least 200 cases before write access is even discussed.

Production writes

Zero. The write path exists in the design and stays disabled until the gate is passed.

This is the pattern I reach for whenever AI wants authority over production: measure it against the humans it would replace, on their real work, and make the promotion criteria explicit before anyone gets attached to the idea.