ADR: Lab UX + Judge panel design

Date: 2026-06-18 Status: Accepted (layout & lean judge panel) · Proposed/pending-confirm (grounding weight ×1)

Context

The experience design must be finalized before writing the build plan. Worked through it interactively with Arijit. Full mockups + spec: repo docs/refactor/ux-design-v1.md.

Decision

Layout (canonical)

• 2×2 panel grid doubles as the scoreboard (no separate matrix-on-top). Rows = retrieval (Keyword top, Neural bottom); cols = architecture (Single left, Multi right). Each panel header carries its own composite score + time; score+time sandwich the grid (keyword bars top, neural bars bottom). Row labels run vertically on the left.
• Panel numbering (row-major): P1 single+keyword · P2 multi+keyword · P3 single+neural · P4 multi+neural (bottom-right = full power).
• Source pills: a dynamic row at the bottom of each panel — one pill per source the answer actually used (count varies; not fixed). Click → open/preview source.
• Judge = slide-in drawer from the right, triggered by clicking a panel's score; shows THAT panel's verdict; pinnable + foldable. Same drawer for all four.

Judge panel content — LEAN (progressive disclosure)

Rule: a block is always visible only if valuable on EVERY answer; contextual → conditional/expander; redundant-with-panel → cut. - Always visible (3): ① Composite + plain verdict + grounding status badge · ②③ Dimensions (Grounding, Confidence, Breadth/Depth — bar + number; rationale on hover) · ⑥ Comparison (the deltas: multi-lift, neural-lift, compound — relative to the opened panel). - Conditional: grounding flagged-claim cards appear only when there's a violation (badge flips ✅→🚫 and expands). - Cut/merge: standalone Sources block (pills already on the panel); keep claim→source mapping inside grounding/“how it answered”. - Expanders: ⑦ How it answered (Maverick trace + real config diff) · ⑧ Per-judge detail (the 3 judges).

Scoring

• 3 dimensions: Grounding, Confidence, Breadth/Depth.
• Blind judge, scored vs each panel's own retrieved sources (fair + credible). → adopt.
• Grounding = hard GATE, weight ×1 (NOT ×2) [RECOMMENDED, pending Arijit confirm]. Rationale: grounding is a floor every panel passes (110% grounded enforced), so ×2 compresses the composite and mutes the breadth/confidence gap that is the experiment's signal. Composite driven by the differentiators (confidence + breadth/depth); grounding gates hallucination + shown for transparency. Differs from existing lab/judge ×2 → reconcile at build.
• Two-tier judging = tied to judge count:
• LIVE = 1 judge, fast model (flash), thin sources → indicative (must self-label). Fast (~20s) for the interactive demo.
• BATCH = 3 judges, strong model (pro), full sources, supermajority grounding gate (claim unsupported only if ≥2/3 agree) → authoritative (the official 2×2 scorecard).
• Two-layer judging: each judge scores all dims (layer 1) → synthesize mean + supermajority (layer 2). Per-judge layer surfaces only in the batch view's "Per-judge detail" expander.

Rationale

Progressive disclosure keeps the default demo-grade while depth is one click away. Grounding ×1 avoids muting the signal we're trying to prove. The 1-vs-3 judge split reconciles demo speed with proof rigor.

Consequences

• Build the score-click → pinnable/foldable judge drawer.
• Reconcile grounding weight with existing lab/judge (currently ×2).
• Question set (recreate) must contain items where neural/multi differentiate, else composite shows "no difference".
• Remaining design piece: the Scorecard view (batch aggregate drill-down from 2×2 to per-question).