Gap Findings — Unified Module Architecture

These 8 gaps were identified during the Perplexity Agent API integration brainstorm. Each needs dedicated design time. Parked for now — will revisit.

Gap 1: The Merge Problem

When a module gets data from multiple sources (BuiltWith says X, Agent API says Y), how do they become one unified output?

Proposed direction: Source-priority merge with conflict preservation. Conflicts are intelligence, not errors. Each field in the Pydantic schema gets a source_priority annotation in the playbook. Merge strategy options: highest_priority_wins, keep_both, flag_for_review.

Key insight: "BuiltWith still detects Elasticsearch but engineering blog says they're migrating to Algolia" is MORE valuable than either data point alone. The schema supports this with fields like current_vendor, migration_signal, and confidence: Literal["confirmed", "conflicting_signals", "single_source"].

Gap 2: Rate Limiter / Queue

Agent API rate limits: Tier 0 = 1 QPS/50 RPM, Tier 2 = 8 QPS/500 RPM. 13 modules running in parallel will exhaust limits immediately.

Proposed direction: Shared async semaphore with priority queue. Singleton class, not a separate service. Priority levels 1-5 based on module importance (intel-company = 1, intel-industry = 5). Auto-scales as cumulative spend crosses tier thresholds.

Gap 3: One Big Call vs. Many Small Calls

Research across prospect + competitors: one Agent API call for all companies, or one per company?

Proposed direction: Playbook declares the strategy. Three patterns: - per-company: One call per company. Higher quality, higher cost. For high-stakes research (search tech, exec quotes). - comparative: One call, all companies. Cheaper. For comparison-native tasks (gap analysis, positioning). - hybrid: Some sections per-company, some comparative.

Gap 4: Citation Validation

Agent API returns URLs as citations, but LLMs hallucinate URLs. Need validation.

Proposed direction: Three-tier validation: - Tier 1: URL existence check (HEAD request, all citations, near-zero cost) - Tier 2: Content verification (fetch page, verify claim exists on page — for high-value claims only: revenue, exec quotes, tech detections) - Tier 3: Cross-source confirmation (compare Agent API claim against BuiltWith/SimilarWeb/Yahoo Finance — zero cost, data already fetched)

Playbook declares which claims are high-value (require Tier 2).

Gap 5: audit-browser Hybrid Approach

Browser-testing 4 competitors = 40+ minutes. Need a faster approach for competitors.

Proposed direction: Browser-test the prospect (high fidelity, Playwright + Claude Vision). Research-test competitors via Agent API (find reviews, tech blogs, demo videos describing their search experience). Unified Pydantic schema covers both: prospect gets actual scores + screenshots, competitors get capability assessments + citations.

Gap 6: Synthesis Modules as Playbooks

synth-business-case and synth-sales-plays don't call external APIs — they read cached upstream results and synthesize with Claude.

Proposed direction: Yes, they become playbooks. The data source is the module cache (FROM CACHE pattern). Playbook instructions tell Claude how to synthesize. Same executor, different data source type.

Gap 7: audit-factcheck Stays Custom

The factcheck module uses the Evaluator pattern — separate agent, raw evidence access, adversarial posture.

Proposed direction: Keep as custom code. It's the quality gate that validates playbook-based modules. Making it a playbook = having the defendant serve as judge. However, standardize its INPUT: every playbook-based module auto-generates a ClaimRegistry (list of factual claims + citations) that factcheck consumes.

Gap 8: Playbook Evaluation

How do you know a playbook produces good results? Need CI for playbooks.

Proposed direction: Design for it now, build Phase 3. - Golden output fixtures (known-good results for test companies like nike.com, costco.com) - Scoring rubric per playbook (completeness, accuracy, citation quality — score out of 30, pass threshold 22) - Eval runner: run playbook on golden company, compare output, score against rubric - Playbook PRs must pass eval before merge

Structure: modules/{name}/evals/golden/*.json + rubric.md + eval_runner.py