wiki/decisions/2026-06-28-grounded-report-qa-gate.md
ADR: Grounded report-QA via a Hermes plugin gate (facts grounded, coaching allowed)
Date: 2026-06-28 Status: Accepted (core verified)
Context
PRISM's defining promise: a rep chats with it and gets answers only from the audit report, never fabricated facts about the prospect. But it must also act as a sales coach (hypotheses, moves), which is reasoning beyond verbatim report text. These two goals conflict unless scoped.
Decision
Enforce grounding with a Hermes plugin (prism-report-qa) registering two hooks, no Hermes
source fork:
- L1 pre_llm_call, injects the bound company's audit-data.json into the turn as the sole
fact source (report bound per session by detecting the company in-message vs the report-store index).
- L4 transform_llm_output, a Gemini grounding judge verifies the drafted answer's
factual claims against the report and rewrites unsupported ones (rewrite-on-fail; fail-closed
on judge error). This hook can rewrite the outgoing message, discovered by reading the image
source (the docs' hook table omitted it).
Scope (load-bearing): the gate polices factual claims about the prospect (numbers, vendor, scores, findings, financials), must be report-supported. It does NOT police coaching (calibrated hypotheses, F1–F6/M1–M10 moves, objection handling, call plans), allowed, as long as the facts they cite are supported. The judge is told this split explicitly.
Rationale
A read-receipt on the live image found transform_llm_output, which mutates output, so a plugin can
hard-gate without forking Hermes (no derived-image/patch maintenance). The judge runs on Gemini
(the working control model), not the ai-judge Claude skill, because Anthropic auth isn't on the box.
Alternatives Considered
| Option | Why rejected |
|---|---|
| Injection + strict prompt only (L1+L3) | Proven insufficient, gemini fabricated "12.5%" (real 15.98%) labeled [FACT] |
Patch agent/turn_finalizer.py (source fork) |
Unnecessary once transform_llm_output hook found; fork = upstream-breakage maintenance |
| Wrapper service in front of Hermes | More infra; the plugin keeps the gate inside Hermes cleanly |
| Strict verbatim-only (no coaching) | Cripples the sales-coach role |
Consequences
- Report-QA verified over the PetSmart report: grounded fact (15.98%, cited real field), absent fact → "That's not in the audit report", coaching allowed + anchored to cited facts.
- Per-turn cost = 2 Gemini calls (answer + judge); needs a billed Gemini key (free tier 429s).
- Hardening TODO: force-a-fabrication stress test; QA-mode tool/delegation lockdown (L2); persist report-binding to a file (currently in-memory).
- Plugin source tracked in repo
docs/workspace/hermes-prism-integration/chowmes-prism/plugins/. Related: 2026-06-28-prism-is-chowmes-prism-hermes-instance.