Dashboard/Algolia-Central (Second-Brain)/v2/wiki/decisions/2026-06-28-drop-expandedquery-from-retrieval.md
Algolia-Central (Second-Brain)

v2/wiki/decisions/2026-06-28-drop-expandedquery-from-retrieval.md

ADR: Drop brain.expandedQuery from retrieval (send the raw user turn)

Date: 2026-06-28 Status: Accepted

Context

The coordinator's brain step rephrased each user turn into a "neural-search-friendly" query (expandedQuery) before retrieval. Backlog A asked whether that rephrase earns its keep, gating its deletion (it's the highest-cost, highest-risk coordinator piece and the documented false-refusal bug pointed at it). Two gates were run on the live CENTRAL app (0EXRPAXB56), AC2_WWW_SINGLE_NEURAL.

Decision

Send the raw user turn to the Agent Studio agent for retrieval; stop passing brain.expandedQuery. brain still runs for intent/entities/proposedQuestion (dossier, baton routing, judge Coverage); expandedQuery survives only as baton's routing-prompt signal — never retrieval.

Rationale

  • Gate 1 (semantic fires on raw NL): proven. Raw conversational queries with getRankingInfo:true return non-zero neuralScore/semanticScore/semanticSimilarity with mergeInfo.semantic present; top hits semantically on-target. NeuralSearch understands raw NL — no rewrite needed for retrieval.
  • Gate 2 (scored A/B, 32 Q): no lift + a grounding hazard. Mean composite Δ +0.24 (within ±0.3 judge noise). The decisive finding: on bait/skeptical queries the rephrase strips the skeptical framing — Q7.5 "…can you confirm the exact figure?" → "Algolia Lacoste case study 150% increase…" → the agent confirmed an unsupported stat (raw refuses 10.0 → rewrite gate-tripped 1.67). Rewrite wins clustered only on vague openers.

Alternatives Considered

Option Why rejected
Keep full intent-rephrase No reliable lift; introduces grounding failures on exactly the bait/refusal class the project must handle
Delete brain entirely Rejected — intent/entities/proposedQuestion still feed dossier, baton, judge Coverage
Keep a narrow turn-≥2 coreference rewrite Deferred (optional) — not needed today; turns are sent fresh + self-contained

Consequences

  • Code: lab/server/src/orchestrate.ts sends userInput (was brain.expandedQuery) at both agent calls; new TDD test "sends the RAW user turn". Shipped on main dc03bc7. Tests 109 green, tsc clean.
  • Evidence: docs/experiment/2026-06-28-expandedquery-drop-validation.md + expandedquery-ab-results.json. Harness lab/server/src/experiments/expandedQueryAb.ts (npm run expt:expandedquery).
  • Lesson: a pre-retrieval rephrase rewrites intent, not just vocabulary — test the bait class. (docs/sop/lessons-log.md.)