Dev Log — ACS

2026-07-10 (post-midnight) — Shipped the classifier fix, killed the orchestrator, fixed 3 real bugs found live

Status: Done, verified live in production. What happened: Full development-loop build (A1-A9) replaced Agent Studio's racy async config.suggestions offer signal with a new dedicated ACS-classifier-neural agent, called synchronously by the client. Architecture Review caught 2 Criticals before Build (a query-provenance risk in the orchestrator design still on the table, and a scopeTools gap). A live 7-pair acceptance probe caught a real classifier over-triggering bug (false offers on ambiguous design questions) before the production flip — fixed and re-verified 7/7 before touching main. The separately-validated pure-orchestrator design (10/10 on real E2E data the night before) was deliberately killed for ACS once Architecture Review's classifier requirement made its routing decision provably redundant — building it would've added 2 network hops to relay a decision the client already has. Kept fully specced for AC2.

After deploying, Arijit actually used the live site and found 3 more real bugs the test suite never caught: an empty-answer flake still triggering the deep-dive offer (fixed — skip the classifier call on blank input); a numbered-list rendering bug where the first fix attempt was too subtle to see (4px spacing — correctly rejected); and the real root cause underneath it, found by directly querying the live DOM (document.querySelector('ol') returned null) — the renderer classified an entire text block as list-or-prose as one unit, so an intro sentence sharing a block with a numbered list sent the whole block to the plain-paragraph path. Fixed by grouping lines into same-kind runs instead. Key decisions: the orchestrator pattern is real, tested, and shelved — not discarded — for AC2's port; don't rebuild it from scratch there, do re-litigate whether AC2's use case actually needs it. mandate-guard.sh's conflict with the auto-mode classifier is now confirmed to also block the classifier's own self-unlock attempt outright (not just the hook) — every push/deploy this session needed Arijit to run it himself, no exceptions. Tests written: A1-A9's full TDD suite (40 vitest assertions by the end), plus 2 live acceptance probes (the classifier precision gate, and the repeated-query caching-race gate) with real captured evidence, not synthetic fixtures. Next: Arijit raised a UI suggestion (lean on existing sample-question chips, add a "click to get started" hint) — not yet scoped. mandate-guard.sh fix still not applied. AC2 port still not started.

2026-07-09 — Corrected the Generic↔Technical handoff design before building it

Status: Pivoted (caught before any code was written — no rework cost) What happened: Started scoping a build to replace the **HANDOFF:technical** text sentinel with the real Agent Studio tool-call mechanism proven in the 2026-07-08 spike (pause Generic, call Technical, resume Generic with the result). Walking the design through with Arijit surfaced that this over-imports a capability the app doesn't need: the frontend already orchestrates both agents directly, so there's no reason for the model itself to autonomously call another agent. The tool-call approach would also have silently broken the "Generic always answers first" UX, since a tool call preempts an agent's own answer text in the same turn (confirmed live in the spike: zero final-answer text when a tool call fires). Corrected design: use Agent Studio's native config.suggestions feature as the offer-detection signal instead, keep the direct Technical call the app already makes today unchanged. Key decisions: ADR: drop tool-call handoff, use native suggestions Tests written: none yet — design-only session, no code touched. Next: wire config.suggestions on both ACS-generic-neural and ACS-technical-neural, remove both prose sentinels (**HANDOFF:technical** and **FOLLOWUP:...**), verify live that a suggestion actually references real retrieved content (not a generic template), and design the multi-turn history accumulation (genericHistory/technicalHistory) before it's needed by real usage.

2026-07-09 (evening) — Build shipped, then a live provider outage + 3 more real bugs found in incident response

Status: Done, verified live What happened: Executed the corrected design above as a 9-task dev-loop (B0-B9): native config.suggestions on both agents, consult_technical_specialist tool removed from Generic, segment-aware history summarization (R11 — a code-health review caught that the naive flatten-then-truncate approach silently deleted Technical's content in realistic-length answers), full test coverage. Deployed to main + Vercel prod.

Then production broke: Google deprecated gemini-2.5-flash-lite mid-session (live 404 from the provider). Every agent + the suggestions config were pinned to it. Root-caused with real browser + network inspection (curl alone gave a false-healthy signal — it doesn't request gzip/HTTP2 the way a browser does, so it never reproduced the resulting protocol error the actual site was throwing). Fixed: switched to gemini-2.5-flash.

Re-testing after that fix surfaced two more real regressions from the SAME DAY's earlier build, not from the model swap: (1) Generic had silently lost its brevity constraint — the deleted HANDOFF section's dead prose was doing double duty as the only thing telling Generic not to write full code itself; (2) both agents recap the previous turn's topic before answering something new, because the "never make the user repeat themselves" instruction plus full conversation history read as license to proactively re-cover old ground. Both fixed in the shared/persona instruction files. Also found and fixed: a sample-question chip's exact wording got cache-poisoned by the day's own heavy repeat-testing (same bug class as 2026-07-03) — reworded. Key decisions: model deprecation risk is now a named, tracked open question (vault open-questions #12) — no monitoring existed to catch it before a user hit it live. Tests written: all of B0-B9's suites (25 vitest + 8 node:test) plus manual live-browser verification for every one of the 4 incident-response fixes — none of these four were validated by API-only testing, each required driving the real deployed site. Next: the mandate-guard.sh vs. auto-mode-classifier conflict (open question #11) blocked every push/deploy tonight and needs a real fix before the next session that touches production. The <Chat> widget swap is still the top deferred architecture item.

2026-07-09→10 (late night) — Arijit called a halt on more building, demanded a real E2E test BEFORE any more code; test built, run, and it passed

Status: Test complete, architecture validated on real data. Build NOT started — explicitly held per Arijit's instruction. What happened: A day of incident response (see above) had produced a written revert plan (docs/plans/2026-07-09-revert-to-tool-call-and-port-to-ac2.md, written 20:34 — status "PLAN ONLY, nothing executed") proposing two options (A: separate classifier call, B: prompt-only ordering fix) to solve the "tool call preempts answer text" problem. Separately, a spike run at 22:16–22:18 (docs/spikes/2026-07-09-pure-orchestrator-VERDICT.md) tested a THIRD, unwritten design — a 2-tool pure-orchestrator agent — using hand-faked tool results, and got 5/5 across 3 rounds, but explicitly flagged as unproven: "nothing proves Generalist can actually produce this [offer] flag in Agent Studio's real response format."

Arijit, angry at a day of "building wrong shit" on assumptions, ordered: stop, don't touch code, build a real E2E test that validates the architecture with REAL data (not fakes), interview him on open design questions first, run the test, only fix code after seeing real results. Interviewed him: confirmed pure-orchestrator is the authoritative design to test, OK to hit live Agent Studio API via disposable -dev copies (established pattern all night), run the full test suite (not staged one-test-at-a-time).

Built scripts/spikes/agent-tool-handoff/e2e-orchestrator-validation.mjs — six test cases (T1 baseline full-text, T2/T5 real offer-signal precision across 5 design + 5 implementation questions, T3 orchestrator chaining fed the REAL generalist output — not a fake — T4 specialist real-answer check, T6 flagged as needing a manual browser pass, out of script scope). Ran it against ACS-generic-neural-dev/ACS-technical-neural-dev (existing, read-only) plus a disposable SPIKE-orchestrator-e2e agent (created, used, deleted — confirmed 204 on delete).

Result: the architecture holds on real data. T1 PASS. T5: 10/10 — the real SPECIALIST:-prefixed suggestion fired on exactly the 5 implementation questions and never on the 5 design questions, closing VERDICT.md's exact open gap (this is a real model output, not a fake). T3: the orchestrator's real chaining decision matched the real offer signal on all 10 trials — BUT the script itself misreported this as "5/10" due to a bug (turn2.tc2 && turn2.tc2.toolName === 'call_specialist' evaluates to null, not false, when no tool call happens, and null === false is false under JS strict equality — a false-negative in the test's own scoring, not an architecture defect). T4: 5/5 real specialist calls returned real grounded answers. Key decisions: none locked yet — Build is explicitly on hold pending Arijit reviewing these results and giving a go-ahead. Tests written: the E2E script itself (e2e-orchestrator-validation.mjs) — has its own bug (documented above) that needs a one-line fix (!!(...) instead of bare &&) before being trusted as a regression gate; the underlying architecture result is correct despite the miscount. Next: (1) fix the test script's null-vs-false comparison bug and re-run to get a clean "10/10" report; (2) run T6 (real browser, network tab) — the one test case not yet covered; (3) reconcile the stale written plan doc (still describes Options A/B, not the now-validated pure-orchestrator design) before any Build task starts; (4) get Arijit's explicit go-ahead before writing any production code — none has been touched tonight.