Dashboard/Algolia-Central (Second-Brain)/spectrum/wiki/open-questions.md
Algolia-Central (Second-Brain)

spectrum/wiki/open-questions.md

Open Questions — ACS (decide before building agents)

  1. Algolia app / index. Reuse CENTRAL app 0EXRPAXB56 with a new Adobe index + ACS- agents, or a fresh app? Trade-off: namespace-in-shared-app (simple, ACS- keeps it disjoint) vs. clean isolation (separate app for a separate customer). — OPEN
  2. Ingest path. Hand-built crawler-army (port AC2 scripts/setup/enrich/) over algolia-central.vercel.app/docs, OR Algolia DocSearch MCP (mcp.algolia.com/1/docsearch/mcp, NeuralSearch, curated DocSet) as the retrieval layer. DocSearch offloads crawl+index+retrieval; does NOT replace grounded-agent + judge. Pilot both on the Adobe docset before committing. — OPEN
  3. Corpus definition. What exactly is "Spectrum" here — Adobe Spectrum design system docs, adobe.com broadly, or the docs served at algolia-central.vercel.app/docs? Confirm the real ingest target. — OPEN
  4. Persona split / source clubbing. AC2 used 9 source values → General/Developer/Marketer. The Adobe corpus may not map the same way. Define after seeing the corpus facets. — OPEN
  5. Repo / git. New standalone repo + remote for ACS, or a branch of AC2? Dir is separate → leaning standalone fork. — OPEN

Base to port from: algolia-central2.

Handoff / suggestions redesign (2026-07-09)

  1. ~~What exactly triggers the "go deeper with specialist?" offer.~~ Resolution (2026-07-09): written into scripts/agents/suggestions_generic.md. Generic's native suggestions.system_prompt emits exactly one suggestion literally prefixed SPECIALIST: (case-sensitive — the frontend parses .startsWith('SPECIALIST:')) only when the question is implementation-heavy: any of "how do I build / implement / create / code / write / use / set up / wire X in React (Spectrum)", or a request for a code example, exact props/types, hooks wiring, TypeScript, event handlers, or version-specific API. For everything else (pure design / overview / when-to-use / "what is X"), it emits an ordinary follow-up with NO prefix. When unsure whether a question is implementation-heavy, it defaults to the SPECIALIST: offer (the user decides whether to take the deep dive). Trigger criteria ported verbatim from the retired HANDOFF section of instructions_generic.md — the same criteria already proven live this session, not re-derived. Technical never emits SPECIALIST: (nothing to hand off to).
  2. ~~Multi-turn context shape sent to Technical on repeated deep dives.~~ Resolution (2026-07-09): send all past questions in full (verbatim) + a SUMMARY of each past answer (not full answer text). Arijit's call — balances context fidelity (specialist always sees the real question history) against token/latency cost (answers compress, since re-reading a full prior code answer verbatim isn't needed to build on it). Applies to both genericHistory and technicalHistory accumulation on 2nd+ rounds. Implementation detail deferred to Build: where the answer-summarization happens (client-side truncation/summarizer call, vs. asking the agent itself to emit a short summary alongside its full answer).

  3. Does genericHistory fold in Technical's past deep-dive answers, or stay Generic-only turns. Leaning yes (so Generic doesn't re-offer or lose context on something already covered), not yet implemented. — OPEN

  4. Future idea (out of scope for current build): Agent Studio's native memory feature. Real platform capability, checked 2026-07-09 — semantic memory (timeless user facts/preferences) + episodic memory (agent's past reasoning), retrieved automatically, persists ACROSS sessions/days (not within one live conversation), requires an X-Algolia-Secure-User-Token tying memories to a real user identity. Doesn't solve within-conversation continuity (that's the stateless-history design above) — but could let ACS remember a returning prospect's past questions across visits. Not needed for this build. — IDEA, not scheduled.

Live incident, 2026-07-09 evening — model deprecation + 3 follow-on bugs

  1. Does Agent Studio's completions API have a cache-bypass mechanism. This project has now hit query-level cache-poisoning twice (2026-07-03, 2026-07-09) on fixed sample-question wording. Rewording works but doesn't prevent a third recurrence on a different fixed string. Worth checking Agent Studio docs/API for a force-fresh header or param. — OPEN
  2. mandate-guard.sh vs. Claude Code's auto-mode classifier — unresolved infra conflict. Arijit's own hook (blocks git push/vercel --prod pending a token-file unlock) and the harness's built-in bypass-pattern classifier fight each other: the classifier vetoes the unlock-file write itself, regardless of Arijit's explicit in-session yes. Every push/deploy the night of 2026-07-09 had to be run manually by Arijit. Two fixes proposed, neither applied: add a Bash allow-rule in ~/.claude/settings.json for the exact pattern, OR redesign the hook to require a real permission-prompt instead of a file-write signal. — OPEN, needs Arijit's decision + implementation next session.
  3. Should agent model IDs be validated/monitored, not just hardcoded. gemini-2.5-flash-lite deprecation broke production with zero warning. No monitoring exists to catch a provider deprecating a model out from under a live agent. Worth a periodic live-completions health check (even a simple cron'd curl) rather than discovering it from a user report. — OPEN, not scheduled.

E2E architecture validation, 2026-07-09→10 late night

  1. ~~Can Generalist really produce a real offerSpecialist-equivalent signal in Agent Studio's real response format (VERDICT.md's flagged open gap)?~~ Resolved 2026-07-10: yes. Real E2E test (docs/spikes/2026-07-10-e2e-orchestrator-results.json, script scripts/spikes/agent-tool-handoff/e2e-orchestrator-validation.mjs) fed the orchestrator agent generalist-dev's REAL SPECIALIST:-prefixed suggestion (not a hand-written fake) across 10 real questions (5 design, 5 implementation). 10/10 correct precision, and the orchestrator's real chaining decision matched the real signal on all 10 (a bug in the test script's own scoring under-reported this as 5/10 — see dev-log entry, root cause is null === false under JS strict equality, not an architecture problem).
  2. Is the pure-orchestrator design (VERDICT.md, validated above) the one to actually build, or does the written revert-plan's Option A/B still apply? The written plan (docs/plans/2026-07-09-revert-to-tool-call-and-port-to-ac2.md) predates this validation and describes different mechanisms. Arijit confirmed pure-orchestrator as authoritative for the test, but the plan document itself hasn't been rewritten to match — needs reconciling before Build starts. — OPEN, next session's first job.
  3. T6 (real browser, network tab, live click-through) never run. Every other test case in the 2026-07-10 E2E suite is API-level. The full 3-agent chain has never been driven from an actual browser session end to end. — OPEN.