PRISM V2 — Open Gaps Before Fable 5 Handoff

Captured 2026-07-05. These are the 10 things standing between "objective is clear" and "a build-ready prompt for Fable 5." Working through them one at a time, in order, starting with #1.

1. Executioner decision itself. 5 candidates (Claude Agent SDK, Managed Agents API, Temporal, Agent Studio-as-executioner, no-engine). Which one, and why — not yet decided.

Progress 2026-07-05, from gap #2's findings: 2 of 5 now closed with hard evidence. - Agent Studio-as-executioner — CLOSED, ruled out. Live-probed (00-manifesto.md §Phase 1): /tools, /teams, /handoffs, /orchestrators, /workflows, /sessions, /memory all 404 against a real Algolia app. No orchestration primitives at all. Confirmed chat-layer fit only. - No-engine — ruled out by definition. It's the status quo that's already failing (Hermes's own monolithic-call problem, see #2 below, is what no-engine would keep). - Remaining 3, re-scored against #2's findings: - Claude Agent SDK — strongest read so far. Directly fixes the diagnosed structural root cause (monolithic single-call → per-AgentDefinition skill dispatch + hooks). Clears the security bar (Anthropic-official, not a 3rd-party community agent like Hermes). Self-hosted — zero ops delta from today's VPS model. Only unresolved weakness: retry/durability (gap #7). - Managed Agents API — real architectural tension, not yet flagged before this pass. Also Anthropic-official (clears security bar) and has native multiagent + Outcomes (a built-in gate mechanism). But it's Anthropic-hosted — the agent loop and the execution container both run on Anthropic's infra, not PRISM's own VPS. The whole point of removing Hermes was getting PRISM running inside Algolia's own VPN; ceding the execution container to Anthropic's cloud is arguably a bigger step away from that goal than Agent SDK's model (self-hosted, only the inference call leaves the VPN — same trust boundary PRISM already crosses via the Claude API today). Needs an explicit gut-check with Arijit before ruling in or out. - Temporal — still theoretically viable, unresolved. Third-party, but a different risk class than Hermes (mature enterprise workflow infra vs. a community agent runtime) — the security blocker doesn't cleanly disqualify it, but no doc has explicitly cleared it either. Would also fix the "no per-skill granularity" root cause (activities = per-skill units) but adds a second orchestration layer Claude's own primitives may already cover — the original 2026-06-28 rejection reason ("redundant alongside Hermes") is dead, but "redundant alongside Agent SDK" is an open question, not yet dead or alive. - DECISION, 2026-07-05: Claude Agent SDK, self-hosted, running on the existing VPS. Reasoning, all from evidence already gathered — no new test needed to call this: - It's Anthropic-official, self-hosted code — clears the security bar (Hermes's problem was being a non-Anthropic community vendor's agent, not self-hosting itself). - It directly fixes the diagnosed root cause: per-AgentDefinition skill dispatch + hooks replaces Hermes's one-monolithic-claude -p-call design. - Its one real gap — durable retry — is now cheap to close: gap #5's decision already puts Postgres in as the state/job-tracking system of record. A job-status table there (job id, step, status, retry count) gives "did this survive a crash and can it resume" without pulling in Temporal as a second orchestration layer. - Managed Agents API is not chosen: it's Anthropic-official too, but it's Anthropic-hosted — the execution container leaves PRISM's own infra, which cuts against the entire reason for this rearchitecture (get PRISM running inside Algolia's own VPN). - Temporal is not chosen: no longer needed once Postgres covers durability: a second full orchestration engine on top of Agent SDK's own primitives would be exactly the "redundant orchestrator" pattern the original 2026-06-28 ADR rejected (that rejection was about redundancy alongside Hermes; the same redundancy argument now applies alongside Agent SDK + Postgres instead). 2. Hermes's actual root cause never diagnosed. ~~Manifesto states "not working as needed, root cause unknown even after investigation."~~ RESOLVED 2026-07-05 — the manifesto's framing was stale. Two functional bugs WERE root-caused and patched 2026-07-01 (memory project-cassandra-observability-gap, docs/sop/lessons-log.md): (a) run-audit.sh's headless claude -p had a default 600s wait ceiling that force-killed the 60+min browser-testing wave with a false-success rc=0 — fixed, proven live; (b) the hermes-prism plugin discarded the runner's real failure reason and returned a hardcoded generic error — fixed in code, live effect still unproven. But the memory itself flags this as only a narrow fix — the real, deeper issue is architectural: Hermes runs the entire audit as one monolithic headless claude -p call, with no per-skill entry points, no per-module completeness checks, no self-heal/re-dispatch, no subagent spawning. That's a structural fit gap, not a mystery bug — and it's exactly what Claude Agent SDK's AgentDefinition + hooks + per-subagent dispatch would fix directly (see finding #3.6 in 03-executioner-claude-agent-sdk-findings.md).

Security rationale, also checked: Hermes = Hermes Agent by Nous Research (github.com/NousResearch/hermes-agent, docs/workspace/hermes-prism-integration/02-hermes-architecture-truth.md:11) — a real, named, third-party open-source community agent runtime, not Anthropic's. The "3rd-party, unchecked, self-hosted-agent — hard security blocker" language exists only in 00-manifesto.md; no separate threat-model doc was found anywhere (checked the 2026-06-28 ADR — no elaboration there either). Implication for #1: the blocker tracks to Hermes being a non-Anthropic community vendor's agent code, not to self-hosting as a concept. Claude Agent SDK (Anthropic's own official open-source SDK, self-hosted) and Managed Agents API (Anthropic-hosted) both clear this bar since both are Anthropic-published. Temporal is third-party too, but a different risk class (mature enterprise infra vs. a community agent framework) — not fully resolved by any doc, worth a direct gut-check with Arijit if it stays a live candidate. 3. Zero live proof-of-concept on any candidate. All evidence so far is doc-read or source-inspected-but-never-executed. 4. Cost/ops comparison never pulled. Self-hosted (Agent SDK, Temporal) vs Anthropic-hosted (Managed Agents) — no numbers against PRISM's actual audit volume. 5. Data backend + state layer — DECIDED 2026-07-05. Verdict: Algolia cannot substitute the database. Split architecture, not a swap.

Ground truth checked first: PRISM's real current store is Postgres (not SQLite) — alembic/versions/001_initial_schema.py through 009_..., prism_platform/db/models.py, docker-compose.yml. Real relational schema: company hierarchy (migration 007), customer evidence (004), knowledge store (008), vertical benchmarks (003), audit data + scores (009) — genuine tables with foreign-key relationships, not a flat file.

Algolia's write/read API (live-tested against a real app, mcp__algolia__*, 2026-07-04 — not just docs): real document-store semantics (saveObject/getObject/batch), fast in practice for single-record writes. But structurally missing what a relational store gives for free: no ACID transactions (every write is an independent async task, no cross-record rollback), no joins (getObjects is a batched key lookup, not a relational join — can't ask "all findings where audit.company=X AND severity=critical" across two related record types in one query), only a filter/facet DSL, not general query. Given PRISM's schema already has real relational structure (company hierarchy, customer evidence tables), Algolia cannot represent that without denormalizing and giving up integrity.

REVISED 2026-07-05 (final, no more caveats): Arijit rejected any dual-layer complexity, and separately reframed the whole project as a sellable, domain-agnostic product (Phase 3), not an Algolia-internal tool — removing the only reason Algolia was ever on the table (see gap #6). Decision: Postgres + pgvector, one system, nothing else. Postgres stays system of record for relational data AND executioner state-tracking (job status, per-step completion, gate pass/fail) — already running, already has the schema, zero new infra. pgvector extension covers semantic/vector search over report content and research corpus, replacing what Algolia's NeuralSearch would have done — one database, not two, and vendor-neutral (a future "YourCMS-PRISM" customer isn't forced into an Algolia contract just to run chat). Algolia has zero role in PRISM's core architecture going forward. It still shows up as domain content — the algolia-search-audit skill audits other companies' search stacks — but that's product content, unrelated to PRISM's own infrastructure. 6. Chat layer — DECIDED 2026-07-05, FINAL. Arijit reframed the whole rearchitecture: this is not an Algolia-internal-sell tool, it's a sellable, domain-agnostic product (Phase 3 vision). Algolia tech use was never supposed to be forced in — only justified by the internal-sell angle, which is now off the table. That kills Agent Studio's only remaining case: confirmed (via existing live research, Algolia-Central2 coordinator findings 2026-06-27) it has mcp_tools as a native tool type, so it COULD call an external MCP server fronting Postgres without duplicating data — but doing so buys zero unique capability over just using Claude Agent SDK directly (no cross-turn memory — proven negative via direct probe, no multi-agent, no orchestration). Verdict: the chat/interactive agent is Claude Agent SDK too — a separate AgentDefinition (different system prompt, read-only tool scope against Postgres/pgvector) running in the same stack as the executioner, not a separate product. One technology end to end, zero Algolia dependency in the core architecture. Ties directly into Phase 2's Researcher/Audit/Synthesizer agent vision — the chat agent is simply a 4th member of that family. 7. Retry/durability requirement — how much do we actually need? No data on actual PRISM audit run durations/failure rates to judge whether Temporal-grade durable execution is load-bearing or overkill. 8. Phase 2 module boundaries undesigned. No written split of which of today's ~15 algolia- skills map to Researcher/Audit/Synthesizer agents. 9. Recipe/cockpit spec doesn't exist. "Operator picks modules = recipe" is one sentence in the manifesto — no schema, storage model, or versioning scheme. 10. Definition of done / test plan missing.* No concrete, testable definition of "working system" to verify Fable 5's output against.

FINAL DECISION, 2026-07-05 — locked

• Database: Postgres + pgvector. One system. Relational integrity for real schema (company hierarchy, customer evidence, audit data, scores), pgvector extension for semantic/vector search over report content and research corpus. Zero Algolia in the core architecture.
• Executioner + operator + monitoring/status-checking + chat interface: all Claude Agent SDK, self-hosted. Not separate technologies for separate roles — a family of Claude agents (AgentDefinitions) under one self-hosted SDK: an orchestrator agent (runs audits, dispatches skill-subagents, enforces mandatory gates via hooks), and a chat/interactive agent (reads Postgres/pgvector, talks to the user) — same stack, different configs. Runs on PRISM's own VPS, calling the Claude API — no execution container ceded to any vendor's hosted infra.

Vendor challenge, 2026-07-05: why Claude Agent SDK over Google or OpenAI? — RESOLVED

Researched with live WebFetch to each vendor's current docs/pricing (not training recall). Honest finding, not a spin job:

• Google ADK (Agent Development Kit) — real, GA ("ADK 2.0, LIVE"), genuinely self-hostable calling only the Gemini API (GCP optional, not required), model-agnostic. Has comparable orchestration primitives: graph workflows, sessions+memory, callbacks (hooks), multi-agent routing, state persistence. The Agent2Agent (A2A) protocol is NOT a competing framework — verified it's a wire-format/interoperability protocol (donated to Linux Foundation), explicitly documented as "not an agent development kit."
• OpenAI Agents SDK — also real, GA ("production-ready"), self-hostable calling only the OpenAI API. Handoffs, sessions, guardrails (validation/blocking), state persistence — comparable primitives to Claude Agent SDK's hooks/SessionStore.
• Capability verdict: neither is disqualified on orchestration grounds. Both are legitimate, mature, self-hostable competitors — ruling them out on "can't do it" would be false.
• Pricing (live-fetched): Gemini 2.5 Pro $1.25/$10 per MTok in/out, Gemini 3.1 Pro (preview) $2/$12. Claude Sonnet 5 (what PRISM actually routes bulk skill-execution to, per its own model-economics table — NOT Opus) is $2 intro/$3 standard input, $10 intro/$15 standard output — essentially a wash against Gemini 2.5 Pro at the tier PRISM actually uses. Claude Opus 4.8 ($5/$25) IS meaningfully pricier than Gemini — Arijit's "isn't Google cheaper" instinct is correct at that tier. OpenAI's GPT-5.5 ($5/$30) is NOT cheaper than Claude — pricier than both Sonnet and Opus on output.
• The actual decisive factor — not price, not capability: PRISM's entire audit engine already exists as 15+ Claude Code Skills (SKILL.md format, executed via claude-cli), tested against real companies — a standing prior decision (memory feedback-prism-orchestrates-skills-not-rebuild: "PRISM orchestrates skills, never rebuilds"). Google ADK and OpenAI's Agents SDK are generic frameworks with their own tool/agent formats — neither runs a Claude Code Skill as-is. Switching means rewriting all 15+ skills from scratch in a different paradigm, for a token-cost delta that's real at the Opus tier but roughly a wash at the Sonnet tier actually used.

FINAL: Claude Agent SDK stays. Explicit caveat, stated honestly: if PRISM were starting from zero with no existing skill library, Google ADK would deserve serious head-to-head evaluation on cost/capability alone. It isn't starting from zero — the switching cost of rebuilding a working, tested product isn't offset by a cost delta that's small at the tier PRISM actually runs on.

Agent design — gap #8, closed 2026-07-05

6 agent roles (Arijit accepted 5+1; the 6th, Gate/QA, was flagged by him as under-designed and redesigned below — it is not one agent):

1. Orchestrator/Executioner — one per audit run. Runs audits, dispatches the pipeline agents below, tracks job state in Postgres, enforces gates via hooks, reports status. Absorbs algolia-search-audit.
2. Researcher — one per audit run, internally fans out. Owns algolia-audit-research (kickoff), algolia-intel-queries (browser-phase prep), and the 12 algolia-intel-* modules (company/competitors/financial-private/financial-public/hiring/industry/investor/news/partner/social/techstack/traffic).
3. Auditor — one per audit run. Owns algolia-audit-browser (live Playwright/stealth browser testing — different tool surface than the reasoning-heavy research skills).
4. Synthesizer — one per audit run. Owns algolia-audit-report, algolia-synth-business-case, algolia-synth-sales-plays, algolia-campaign-abx.
5. Chat — standing/long-lived, not per-run. Reads Postgres/pgvector directly, answers user questions grounded in published audit data.

6. Gate/QA — redesigned 2026-07-05: a Verification Pipeline, not one agent

Arijit's correction: this is the single highest-stakes piece in the whole system — the data this product produces has zero tolerance for fabrication, but full tolerance for omission (missing data is acceptable; wrong data is a multi-million-dollar legal exposure). That asymmetry drives the whole design: every stage below defaults to strip the claim on any doubt, never publish on doubt — consensus is required to KEEP a claim, not required to REJECT one.

5 stages, run per-module (not once at the end — this is the exact gap already diagnosed in the manifesto's Phase 2 task list):

1. Mechanical Validator (deterministic script, no LLM) — schema validation, does every citation URL actually resolve, does every [FACT] tag have a source, required fields present. Free, instant, zero hallucination risk of its own.
2. Fact-Check Agent (existing algolia-audit-factcheck) — verifies claims against cited sources, confirms quotes match transcripts verbatim, cross-file consistency.
3. Adversarial Verifier Panel — NEW, the redundancy Arijit asked for. 3 independent agents, different lens each, run in parallel, none sees the others' verdicts: Numbers Auditor (does every stat trace to a specific cited source), Quote Auditor (is every quote verbatim, not paraphrased-and-attributed), Skeptic (actively tries to refute — job is to find ANY unsupported claim, not confirm). This is the piece that was missing — self-grading is weak; independent adversarial verification is the real redundancy pattern for this stakes level.
4. Quality/Completeness Agent (existing algolia-audit-eval) — depth, coverage, instruction-adherence. Lower-stakes than fabrication — a different concern ("is this good enough" vs "is this true").
5. Final Legal/Liability Gate — runs ONCE on the fully-assembled deliverable, after 1–4 pass per-module. Holistic cross-file check across all 6 deliverable files. Explicit framing: "would this survive a lawyer's review before it reaches a prospect," not just internal consistency.

Model tier: Researcher/Auditor/Synthesizer run Sonnet (bulk work, per PRISM's own routing table). Adversarial Verifier Panel + Final Legal Gate run Opus — per the existing model-economics rule ("severity escalates a tier" for expensive/irreversible mistakes). This is the one place in the system where cost-optimizing down would be the wrong call.

Mechanical wiring (Claude Agent SDK primitives, already confirmed live): each of Researcher/Auditor/Synthesizer's SubagentStop hook fires stages 1–4 before that module's output is accepted; decision:"block" forces revision on failure. Stage 5 runs once, orchestrator-triggered, after all modules pass, before anything is published or sent to an AE.

Status: gaps #1/#5/#6/#8 closed and locked above. Remaining open: #3 (proof-of-concept, build-verification not blocking design), #4 (cost/ops sizing), #9/#10 (recipe/cockpit spec, definition of done), #11/#12/#13 (below, added 2026-07-06).

Gaps #11-13 — added 2026-07-06, Arijit's correction

Everything above is org-chart level: which agents exist, what pipeline stage they belong to. None of it is runtime-mechanics level. Arijit's framing: this is "the heart and soul of the system" — screens are surface, this is the operation underneath. Not optional before Fable 5 since the build target is frontend + backend + middleware together, not screens alone.

11. Agent-to-agent communication protocol — undesigned. No spec exists for:
    • The exact instructions/system-prompt each agent receives from the orchestrator per invocation (not "Researcher owns these skills" — the literal prompt text and how it's parameterized per audit).
    • The data contract for what comes back from each agent — is it structured JSON against a defined schema, free text, a file path? No schema exists for any agent-to-agent handoff today.
    • Sequencing logic: is the pipeline fixed-order (Researcher → Auditor → Synthesizer) or can the orchestrator branch/re-dispatch based on what comes back? Gap #8 assumes fixed order; never stated explicitly or tested.
    • Message-level error/retry: gap #7 covers job-level durability (did the whole run survive a crash); this is one level down — if Researcher's output fails the mechanical validator, does the orchestrator re-prompt the same agent, re-dispatch fresh, or halt the run? Not decided.
12. Multi-tenancy architecture — undesigned. Everything decided so far (IA doc, 05-role-driven-ia.md) is explicitly single-operator: "internal tool, not multi-tenant yet." If this becomes Algolia's sales team (~200 people) or a sellable product (Phase 3), needed and currently absent:
    • Per-user auth/login and session management at scale (distinct from the role-filter-not-ACL decision already made — that's role-within-one-tenant, not tenant-to-tenant isolation).
    • Data partitioning — does tenant A's audit data live in the same Postgres tables as tenant B's, gated by a tenant_id column, or physically separate? No schema decision made.
    • Concurrency/rate-limit model — 200 people hitting Claude Agent SDK simultaneously means queueing, per-tenant rate limits, and cost attribution per org. Nothing here today; single-operator load never required it.
13. Security/deployment topology — undesigned. What's behind a firewall vs public, the network boundary between chat agent / orchestrator / Postgres, the auth provider (open question, same bucket as the TTS vendor pick), secrets/key management per-tenant once this is sellable, and audit logging if an enterprise buyer asks for it. None of this has a decision or even a candidate list yet.

MAJOR CORRECTION, 2026-07-06 ~2:45am — read before touching #11 again. The draft below was written without checking docs/workspace/hermes-prism-integration/phase-b-cass-agent/L2-execution-orchestration-design.md (2026-06-29, status LOCKED on hybrid routing) — a much deeper, already-decided design for exactly this problem, for the CURRENT V1 (Hermes) system. It reframes the whole question correctly: "gemini tool-calling as the execution model" is the wrong lens — PRISM execution is multi-agent orchestration, Cass is a thin conversational front, not the orchestrator. It has a locked 3-tier runtime model (script/$0, research/Perplexity, heavy/claude-cli), a real worker-agent roster mapped against the actual 17-module registry, and a concrete build order. Live-verified tonight (VPS SSH, not just reading code): - run_audit and audit_status tools ARE built and ARE registered live (plugin.yaml: provides_tools: [run_audit, audit_status]) in the currently-deployed hermes-prism container's prism-report-qa plugin. - The backend they call (prism-runner on 127.0.0.1:8770, plus prism_platform's FastAPI audits.py/audit_stream.py) is real and PROVEN — /jobs shows actual completed runs (belk published score=5.3, dell published score=2.7), not stubs. - What's NOT proven: no log evidence in the last 7 days of these tools being invoked through an actual chat conversation. The wiring exists; whether it works end-to-end when a real person types "run an audit on X" into Cassandra has not been demonstrated. - Plugin drift resolved: confirmed docs/workspace/cassandra-tooling/live-sources/prism-report-qa__init__.py is byte-identical to the live VPS copy (verified via diff). docs/workspace/hermes-prism-integration/chowmes-prism/plugins/prism-report-qa/__init__.py is STALE — missing two real fixes shipped 2026-07-03 (sticky report-binding bug, running-token-collision bug). Don't read from the chowmes-prism/plugins/ copy going forward; cassandra-tooling/live-sources/ is canonical. - Real remaining gap, scoped correctly now: not "design the protocol" (already designed, L2-execution-orchestration-design.md §3-6) — it's (1) prove the chat-triggered path works live, and (2) build "notify" (proactive push when a job finishes) — today's status-check is pull/poll-only (audit_status tool, or SSE while a stream is open); nothing pushes to the user unprompted. That's the one piece with no design AND no code anywhere found tonight.

Sequencing note (superseded framing, kept for history): #11 is the most urgent of the three — it blocks writing a real build-ready prompt for Fable 5 regardless of tenant count, since even a single-operator system needs agent-to-agent contracts defined. #12/#13 matter the moment this is sold beyond one operator (Algolia sales team or Phase 3 product) but don't block a first working single-tenant build. Recommend designing #11 alongside the remaining screen reviews (parallel tracks, not sequential), and treating #12/#13 as a defined-but-deferred phase gate — explicit enough that Fable 5 doesn't accidentally build a single-tenant-only system that can't be retrofitted.

UPDATE 2026-07-06 ~2am: Arijit confirmed multi-tenancy must be designed from day one, not deferred (see SESSION.md same date). #12 first-pass shape drafted below — DRAFT, awaiting his approval, nothing built yet.

Gap #12 draft proposal — multi-tenant data shape

Three standard approaches, evaluated against PRISM's already-locked architecture (single Postgres + pgvector, no dual-layer complexity — gap #5's decision):

1. Shared tables + tenant_id column + Postgres Row-Level Security (RECOMMENDED). Add a tenant_id to existing tables, enforce isolation with RLS policies at the database layer (not just careful app code). Cheapest retrofit onto the schema that already exists (alembic/versions/001...009) — one migration, not a rebuild. RLS means a bug in application code can't leak tenant A's data to tenant B; the database itself refuses the query. Right bar for "the system 200 people depend on daily." - Trade-off: all tenants share tables/indexes (noisy-neighbor risk at real scale), and it doesn't give contractually-provable physical data separation if a future enterprise customer demands it (Phase 3 concern, not a V2 blocker).
2. Schema-per-tenant (Postgres supports multiple schemas in one database). Stronger isolation than RLS without separate DB servers. Rejected for V2: migrations run N times (once per tenant schema) and this doesn't stay simple past a few dozen tenants — real ops burden for no benefit at Algolia-internal scale (one tenant to start, maybe a handful if sold to other Phase 3 customers).
3. Database-per-tenant. Strongest isolation, heaviest ops burden. Rejected for V2: overkill for an internal tool; revisit only if Phase 3 lands an enterprise customer that contractually requires physical data segregation.

Auth provider — still genuinely open, not guessed at: does Algolia already have an SSO/OIDC provider (Okta, Google Workspace, etc.) PRISM should integrate against for its ~200 internal users, or does this start with simple email/password? This is a real question for Arijit, not something to invent — same bucket as the undecided TTS vendor.

Concurrency: RLS + one Postgres instance handles data isolation; it does not by itself solve 200 concurrent users hitting Claude Agent SDK at once (rate limits, queueing, cost attribution per tenant) — that's a separate, still-open piece of #12, not covered by the data-shape decision above.