Algolia-Search-Audit
Vault: myos Path: Algolia-Search-Audit
Index.md
Algolia Search Audit — Project Index
An automated intelligence + evaluation pipeline that audits a prospect's website search experience, produces a scored report with evidence, and generates all sales enablement deliverables for Algolia AEs.
Status: Operational but fragile — requires 4-6 hours manual shepherding per audit
Goal: Fully automated, one-command, ~30 min pipeline with observability
Location: ~/.claude/skills/algolia-search-audit/
Workspace Convention: $ALGOLIA_AUDIT_DIR/{CompanyName}/
Scout relationship: Scout is the acquisition/evidence layer for web evidence when audit modules need citable source capture, structured page extraction, product/company/careers/investor/news records, browser/session capture, or portable artifacts. Algolia Search Audit remains the interpretation and deliverable pipeline.
What It Does
Given a prospect domain (e.g., dsw.com), the system:
- Collects intelligence — company context, tech stack, traffic, competitors, financials, investor quotes, hiring signals, social/news signals, partner ecosystem, industry benchmarks (11 modules, parallel)
- Generates test queries — 14-18 typed search queries based on vertical + product catalog
- Audits live search — Playwright browser automation tests autocomplete, NLP, typos, facets, empty states, merchandising (20 test steps with screenshots)
- Synthesizes business case — 6-component ROI model with all formulas shown
- Generates sales plays — AE playbook with SPIN questions, MEDDPICC map, objection handling
- Renders deliverables — SPA report, AE action report, battle card, leave-behind, PDF, ABX campaign
- Factchecks — 20-dimension quality gate before publish
- Publishes — to GitHub Pages via Vercel
Architecture Overview
algolia-search-audit (SKILL.md)
│
├── AGENT-CONTEXT.md ← canonical field names, CSS classes, tokens
├── platform.config.json ← model assignment per task type
├── templates/
│ ├── audit-data.schema.json ← JSON Schema for final output
│ ├── index-template.html ← SPA report template
│ ├── ae-action-report-template.html
│ ├── strategic-battle-card-template.html
│ └── prospect-leave-behind-template.html
├── scripts/
│ ├── collect-company.py ← Python data collection
│ ├── collect-techstack.py
│ ├── collect-traffic.py
│ ├── collect-competitors.py
│ ├── collect-financials.py
│ ├── collect-hiring.py
│ ├── collect-social.py
│ ├── collect-news.py
│ ├── collect-industry.py
│ ├── collect-investor.py
│ ├── collect-exec-media.py
│ ├── calculate-roi.py ← deterministic ROI formulas
│ ├── calculate-score.py ← scoring algorithm
│ ├── generate-audit-data.py ← assembles final JSON
│ ├── parse-builtwith.js ← filters 190KB BuiltWith response
│ ├── publish-audit.sh ← staging + GitHub push
│ └── audit-browser.js ← Playwright browser automation
└── Sub-skills (20 total, invoked via Claude Code Skill tool):
├── Wave 1: 11 intelligence modules (parallel)
├── Wave 2: 1 query generation module
├── Layer 2: 1 browser audit module
├── Layer 3: 4 synthesis modules (sequential)
└── Layer 4: 1 factcheck + 1 eval module
Wave Execution Order
Wave 1 — Intelligence Collection (all parallel, no dependencies)
| # | Module | Skill Name | Model | Python Script | Output Files |
|---|---|---|---|---|---|
| 1A | Company Context | algolia-intel-company |
Haiku + enrichment | collect-company.py |
01-company-context.md/.json |
| 1B | Tech Stack | algolia-intel-techstack |
Programmatic | collect-techstack.py |
02-tech-stack.md/.json |
| 1C | Traffic | algolia-intel-traffic |
Programmatic | collect-traffic.py |
03-traffic-data.md/.json |
| 1D | Competitors | algolia-intel-competitors |
Haiku + enrichment | collect-competitors.py |
04-competitors.md/.json |
| 1E | Financial (Public) | algolia-intel-financial-public |
Haiku + enrichment | collect-financials.py --ticker X |
08-financial-profile.md/.json |
| 1F | Financial (Private) | algolia-intel-financial-private |
Haiku + enrichment | collect-financials.py --private |
08-financial-profile.md/.json |
| 1G | Investor Intel | algolia-intel-investor |
Opus | collect-investor.py |
11-investor-intelligence.md/.json |
| 1H | Hiring Signals | algolia-intel-hiring |
Programmatic | collect-hiring.py |
09d-hiring-signals.md/.json |
| 1I | Social Signals | algolia-intel-social |
Programmatic | collect-social.py |
09b-social-signals.md/.json |
| 1J | News Signals | algolia-intel-news |
Programmatic | collect-news.py |
09c-news-signals.md/.json |
| 1K | Partner Intel | algolia-intel-partner |
Sonnet | None | 07-partner-intel.md (no JSON!) |
| 1L | Industry Intel | algolia-intel-industry |
Opus | collect-industry.py |
06-industry-intel.md/.json |
Gate 1: All 12 .md files exist, each > 500 bytes. Run 1E OR 1F (not both).
Wave 2 — Query Generation (depends on 1A + 1B)
| Module | Skill | Model | Output |
|---|---|---|---|
| Query Set | algolia-intel-queries |
Opus | 05-test-queries.md |
Gate 2: File exists, > 500 bytes, contains ≥14 queries across 8 types.
Layer 2 — Browser Audit (depends on Wave 1 + 2)
| Module | Skill | Model | Output |
|---|---|---|---|
| Browser Testing | algolia-audit-browser |
Opus | 09-browser-findings.md + screenshots/*.png |
Uses Playwright with stealth mode. 20 test steps covering autocomplete, NLP, typos, facets, empty states, merchandising. Takes screenshots as evidence.
Gate: ≥ 10 screenshots, findings file > 2000 bytes.
Layer 3 — Synthesis (sequential, each feeds next)
| Step | Module | Skill | Model | Reads | Output |
|---|---|---|---|---|---|
| 3A | Business Case | algolia-synth-business-case |
Sonnet | financials, competitors, browser | {slug}-business-case.md |
| 3B | Sales Plays | algolia-synth-sales-plays |
Opus | investor, competitors, hiring, financials | {slug}-playbook.md |
| 3C | Report + Deliverables | algolia-audit-report |
Opus | ALL research/*.md | {slug}-audit-data.json + all HTML |
| 3D | ABX Campaign | algolia-campaign-abx |
Opus | research/* + audit-data.json | abx-campaign/ folder |
Gate: audit-data.json + index.html exist.
Layer 4 — Quality + Publish
| Module | Skill | Output |
|---|---|---|
| Factcheck | algolia-audit-factcheck |
FACTCHECK_GATE.md (PROCEED / WARN / BLOCKED) |
| Eval | algolia-audit-eval |
{skill-name}-eval-report.md |
Workspace Directory Structure
$ALGOLIA_AUDIT_DIR/{CompanyName}/
├── research/
│ ├── 01-company-context.md
│ ├── 01-company-context.json
│ ├── 02-tech-stack.md
│ ├── 02-tech-stack.json
│ ├── 03-traffic-data.md
│ ├── 03-traffic-data.json
│ ├── 04-competitors.md
│ ├── 04-competitors.json
│ ├── 05-test-queries.md
│ ├── 06-industry-intel.md
│ ├── 06-industry-intel.json
│ ├── 07-partner-intel.md ← NO JSON (known gap)
│ ├── 08-financial-profile.md
│ ├── 08-financial-profile.json
│ ├── 09-browser-findings.md
│ ├── 09b-social-signals.md
│ ├── 09b-social-signals.json
│ ├── 09c-news-signals.md
│ ├── 09c-news-signals.json
│ ├── 09d-hiring-signals.md
│ ├── 09d-hiring-signals.json
│ ├── 11-investor-intelligence.md
│ ├── 11-investor-intelligence.json
│ └── FACTCHECK_GATE.md
├── deliverables/
│ ├── screenshots/ ← browser audit evidence
│ ├── {slug}-audit-data.json ← final assembled output
│ ├── {slug}/
│ │ ├── index.html ← SPA report
│ │ ├── ae-report.html
│ │ ├── strategic-battle-card.html
│ │ └── prospect-leave-behind.html
│ ├── {slug}-business-case.md
│ ├── {slug}-playbook.md
│ └── abx-campaign/
│ ├── email-sequence.md
│ ├── linkedin-messages.md
│ └── loom-script.md
└── audit-progress.jsonl ← append-only progress log
Canonical Field Names
Defined in AGENT-CONTEXT.md. The final audit-data.json has 29 top-level fields:
meta | cover | score | company_snapshot | executives | intelligence_signals | competitors | findings | gap_pairs | toc | financials | traffic | tech_stack | ae_fields | next_steps | methodology | bibliography | competitive_synthesis | golden_angle | strategic_angles | hiring | icp_mapping | abx_sequence | case_studies | demos | partner_intel | tab_subtitles | recommended_first_play | industry_context
Each finding has 15 fields: id | title | severity | category | tested_query | expected_behavior | actual_behavior | impact_stat | impact_stat_source | screenshot_file | prospect_description | algolia_solution | algolia_case_study_url | algolia_case_study_company | algolia_case_study_result
Score breakdown has 10 dimensions: latency | typo_tolerance | query_suggestions_empty_state | intent_detection | merchandising_consistency | content_commerce_ux | semantic_nlp_search | dynamic_facets_personalization | recommendations_merchandising | search_intelligence
Source Labeling Convention
Every data point must carry a provenance label:
- [FACT — {source} {method}, {date}] — verified first-party data
- [ESTIMATE — {source}, {date}] — inferred/modeled (all private company financials)
- [WEBFETCH — {source}, {date}] — scraped from page
- [WEBSEARCH — {query}] — found via search
Evidence tier hierarchy: AUTHENTIC > WEBFETCH > WEBSEARCH > NO_SOURCE (drop entirely)
MCP Servers Used
| MCP Server | Used By | Purpose |
|---|---|---|
| BuiltWith | techstack, competitors | Domain tech detection (7 endpoints) |
| SimilarWeb | traffic, competitors | Traffic data (11 endpoints), similar sites |
| Yahoo Finance | financial-public, investor | Stock info, financials, news |
| Apify | hiring, social, news | LinkedIn jobs, social scraping, Google News |
| Crossbeam | partner | Partner overlap detection |
| Chrome DevTools / Playwright | browser | Live search testing |
Related
- AgentHarnessPatterns — Anthropic's recommended harness patterns
- Module-Catalog — detailed per-module assessment
- Known-Issues — current failure modes
- Refactor-Architecture — planned Python harness
- Manifesto — module contract patterns