AC2-RC2 — Neural RC2-Replica, Reference-Graded vs RC2, Loop-Refined

Decision (Arijit, 2026-06-20/21). Pivot away from the keyword×architecture 2×2. Build a neural-only product that reproduces RC2's quality, depth, grounding, and engagement on full Algolia infrastructure (Algolia neural search + Agent Studio agents + a thin Algolia-coordinating layer) and adds the grading layer RC2 lacks. Repo design spec is the source of truth: docs/superpowers/specs/2026-06-21-ac2rc2-neural-rc2-replica-design.md.

Context / why

• RC2 (the all-custom POC) gives amazing answers everyone loves, but they are NOT earned on Algolia's stack. RC2 = the quality floor, not the infra floor. Keyword search is no longer the leadership story — Algolia leads with neural/natural-language.
• RC3 (the prior attempt to move RC2 onto Agent Studio) fell short: baton transfer + discovery incomplete, and — per its own brief — it had no captured RC2 traces to build against. The gold-capture step below is precisely that missing foundation.

The decisions

1. Keyword dropped; contest = neural single-agent vs neural multi-agent. Success = BEAT RC2, not just match.
2. Measurement = reference-based. RC2's captured answer is the answer key. Human-driven RC2 capture → fixed replay script → automated AC2 replay (turns align 1:1; AC2's own follow-up graded but doesn't steer the path).
3. Gold seed = RC2's 8 "V3" scenarios (rc2-algolia/e2e/v3-questions.spec.ts). Expand to 50–100 later → train/held-out split.
4. Judge = 7 reference-anchored criteria: coverage · depth · on-point-for-user · right-expert · good-next-question · grounded (hard floor, code check) · voice (vs RC2's voice, partly code check). Headline "% of floor"; exceeding rewarded. Validate the judge against Arijit (~80% on 30–50 cases) BEFORE any loop. Cross-provider judge for the reward role (open ADR vs same-provider rule).
5. Loop = build the gym, then train. Runnable skeleton + replay + validated judge + gold first; loop refines prompts first (coordination/retrieval human-gated); no-regression keep/rollback. Don't loop architecture on a soft judge. Grounded in docs/research/2026-06-21-loop-driven-eval-driven-methodology.md (Karpathy verifiability thesis, Cherny loop-engineering, Anthropic eval-driven doctrine).
6. Engine = RC2-shape routing (NOT fan-out): Maverick hub → Onion-Protocol discovery → classify need → baton to the ONE fitting specialist → deep-dive → yield. v1 cast = ported RC2 personas (Maverick/Elena/Bruno; Maverick's personality retained) from rc2-algolia/config/system/PERSONAS.md. Specialist = persona-lens + source-facet charter. Finer/source-based specialists = a measured hypothesis vs the 3-cast.
7. "Everything Algolia" = Algolia core + thin glue. Retrieval = neural; all agents = Agent Studio; content = Algolia-indexed; custom code only as a thin coordinator.
8. GTM destination: same engine on a per-company audit corpus, fronted by a chat surface (Telegram) an AE talks to. Later, after the engine is proven in RC2's domain.

Consequences / open

• Replaces the parallel content-source fan-out and the keyword panels from the 2×2 ADRs.
• Open ADRs to record: cross-provider judge for reward; DoD thresholds; LLM provider (OpenAI billing-blocked vs Gemini); naming (AC2-RC2 vs RC4); Maverick's home; the product UI/surface (not yet designed).
• Working dir consolidated to /Users/arijitchowdhury/Dropbox/AI-Development/RAG/Algolia-Central2 (the -OLD path is abandoned).
• Next: writing-plans scoped to Phase 0 (capture gold) + Phase 1 (build the gym).