Algolia-Search-Audit

Module-Catalog.md

Algolia Search Audit — Module Catalog

Detailed reference for all 20 modules. For each: what it does, inputs, outputs, what's scripted vs LLM, current gaps, and codification opportunities.

Wave 1 — Intelligence Collection (11 modules, parallel)

1A: Company Context (`algolia-intel-company`)

Purpose: Foundation module. Collects company overview, vertical classification, executives, portfolio/sub-brands. All downstream modules depend on this.

Execution Model: Python script (collect-company.py) + LLM enrichment

Aspect	Detail
Script	`collect-company.py <domain> <output-dir> [--company-name X] [--ticker X]`
Script does	BuiltWith keywords-api for SEO meta, WebFetch `/about` pages, basic company fields
LLM enriches	Executives (names, titles, LinkedIn), vertical classification, HQ/founded/employee_count, Twitter handle, IR URL, portfolio/sub-brand detection
MCP required	BuiltWith, WebSearch
Output	`01-company-context.md` + `01-company-context.json`
Reads from	Nothing — always runs first
Gate	Both files exist, `.md` ≥ 1500 bytes, JSON valid with `meta.skill_enrichment_completed = true`, `company_name`/`domain`/`vertical` not null, ≥1 executive

Key JSON fields: company_name, domain, vertical, business_model, hq, founded, employee_count, public_private, ticker, linkedin_url, twitter_handle, ir_url, executives[], primary_market, parent_entity, is_conglomerate, portfolio_brands[]

Known Issues: - Executive deduplication missing — same person can appear twice from different sources - Portfolio detection (Step 3) uses unstructured LLM web scraping — could be a more structured fetch pattern - vertical classification is LLM judgment — no fixed taxonomy enforced - No Pydantic schema to validate the JSON output

Codification Opportunity: - Vertical classification → enum with keyword matching + LLM fallback - Portfolio detection → structured WebFetch of 3 known paths + Tavily, parse with rules

1B: Tech Stack (`algolia-intel-techstack`)

Purpose: Detects current search vendor, ecommerce platform, analytics, CDN/WAF. Critical because it determines: displacement (they have a competitor), expansion (they have Algolia), or greenfield.

Execution Model: Python script (collect-techstack.py) + LLM enrichment

Aspect	Detail
Script	`collect-techstack.py <domain> <output-dir>`
Script does	All 7 BuiltWith endpoints + `parse-builtwith.js` filter (190KB → <15KB). SimilarWeb tech endpoint.
LLM enriches	`search_vendor_status` classification (ACTIVE_LAYER1 / TAG_ONLY / REMOVED / UNDETECTED), SimilarWeb cross-check interpretation
MCP required	BuiltWith (7 endpoints), SimilarWeb (technologies)
Output	`02-tech-stack.md` + `02-tech-stack.json`
Reads from	`01-company-context.json` (domain only)
Gate	Both files exist, `.md` ≥ 2000 bytes, `search_vendor` not null, `ecommerce_platform` not null

Key JSON fields: search_vendor, search_vendor_status, algolia_detected, ecommerce_platform, analytics_stack, cdn_waf, tech_stack_summary, removed_technologies[]

Known Issues: - search_vendor_status classification is informal LLM judgment — should be deterministic rules - Layer 2 network confirmation happens in browser audit but skill doesn't reference this connection - No Pydantic schema

Codification Opportunity: - Status classification → deterministic rules: BuiltWith present + SimilarWeb confirms = ACTIVE_LAYER1, BuiltWith only = TAG_ONLY, etc. - Entire module could be 100% Python (no LLM needed)

1C: Traffic & Engagement (`algolia-intel-traffic`)

Purpose: Full traffic profile — visits, bounce rate, devices, traffic sources, geography, keywords, audience interests, rank.

Execution Model: Python script (collect-traffic.py) — fully programmatic

Aspect	Detail
Script	`collect-traffic.py <domain> <output-dir>`
Script does	All 11 SimilarWeb endpoints
LLM enriches	WebSearch fallback when `sources_succeeded < 3`
MCP required	SimilarWeb (all 11 endpoints)
Output	`03-traffic-data.md` + `03-traffic-data.json`
Reads from	Nothing
Gate	Both files exist, `meta.skill_enrichment_completed = true`

Key JSON fields: monthly_visits_raw, bounce_rate, pages_per_visit, visit_duration, top_channels[], device_share, demographics, source, source_url

Known Issues: - WebSearch fallback is LLM-driven but follows a pattern that could be scripted - Degraded mode JSON structure defined but not enforced with schema - sources_succeeded counter logic is in the skill description but may not match script implementation

Codification Opportunity: - Already 95% codified. Just needs: Pydantic schema + scripted fallback logic

1D: Competitors (`algolia-intel-competitors`)

Purpose: Identifies who competes with the prospect, detects their search tech via BuiltWith, finds Algolia customers in the same vertical ("Golden Angle"), classifies competitive scenario.

Execution Model: Python script (collect-competitors.py) + heavy LLM enrichment

Aspect	Detail
Script	`collect-competitors.py <domain> <output-dir>`
Script does	SimilarWeb similar-sites, keyword-competitors
LLM enriches	BuiltWith lookup per competitor, Golden Angle detection (algolia.com/customers scraping), competitive scenario classification, case study matching
MCP required	SimilarWeb, BuiltWith (per competitor), WebSearch, WebFetch
Output	`04-competitors.md` + `04-competitors.json`
Reads from	`01-company-context.json`, `02-tech-stack.json`
Gate	Both files exist, `competitive_scenario` not null, ≥2 competitors

Key JSON fields: competitors[] (name, domain, search_vendor, search_vendor_status), competitive_scenario (GOLDEN/DEFENSIVE/OFFENSIVE/MIXED), golden_angle.competitors_using_algolia[], case_studies[]

Known Issues: - Algolia customer discovery scrapes algolia.com/customers every run — should be a static dataset - BuiltWith per-competitor is LLM-driven (should be batched Python) - meta vs _meta bug in skill text (literal typo: "use meta key, NOT meta") - Case study matching is LLM guesswork — should use a curated case study index - Competitive scenario classification is informal — should be deterministic rules

Codification Opportunity: - BuiltWith per-competitor → batch Python script - Algolia customer list → static algolia_customers.json refreshed monthly - Scenario classification → deterministic: any competitor uses Algolia = GOLDEN, prospect uses Algolia = DEFENSIVE, etc.

1E: Financial Profile — Public (`algolia-intel-financial-public`)

Purpose: 3-year revenue trend, EBITDA, margins, analyst consensus, SEC 10-K digital revenue data, earnings call executive quotes.

Execution Model: Python script (collect-financials.py --ticker X) + LLM enrichment

Aspect	Detail
Script	`collect-financials.py <domain> <output-dir> --ticker X`
Script does	Yahoo Finance MCP — all endpoints (stock info, financial statements, recommendations, historical prices, news)
LLM enriches	SEC EDGAR 10-K WebFetch (digital revenue, tech investment mentions), earnings call transcript extraction, `margin_zone` classification
MCP required	Yahoo Finance (all endpoints), WebFetch, WebSearch
Output	`08-financial-profile.md` + `08-financial-profile.json`
Reads from	`01-company-context.json` (ticker)
Gate	Both files ≥ 5000 bytes, `revenue_fy2025` at top level, ≥3 `[FACT — Yahoo Finance MCP` labels, `margin_zone` + `roi_formula_shown` at top level

Key JSON fields (MUST be top-level, not nested): revenue_fy2025, revenue_fy2024, revenue_fy2023, margin_zone (RED/YELLOW/GREEN), roi_formula_shown (boolean), financials{}, executive_quotes[], analyst_consensus

Known Issues: - margin_zone is pure math (>40% GREEN, 20-40% YELLOW, <20% RED) but done by LLM — often wrong - roi_formula_shown is frequently omitted (skill mentions it as CRITICAL twice) - Top-level key requirement for revenue fields is frequently violated — LLM nests them under financials.* - SEC EDGAR fetching overlaps with 1G (investor intel) — same data fetched twice - No shared utility for SEC/earnings data

Codification Opportunity: - margin_zone → 3-line Python function - roi_formula_shown → default true in schema - SEC EDGAR fetch → shared collect-sec-data.py used by both 1E and 1G - Revenue field position → Pydantic schema enforces top-level placement

1F: Financial Profile — Private (`algolia-intel-financial-private`)

Purpose: Revenue estimation for private companies via 6-source waterfall.

Execution Model: Python script (collect-financials.py --private) + heavy LLM enrichment

Aspect	Detail
Script	`collect-financials.py <domain> <output-dir> --private`
Sources	ecdb.com, PitchBook/Crunchbase, LinkedIn headcount, trade press, Inc 5000/Deloitte Fast 500, job posting volume
All figures labeled	`[ESTIMATE]` — never `[FACT]` for private companies
Output	`08-financial-profile.md` + `08-financial-profile.json`
Gate	Both files ≥ 3000 bytes, `revenue_confidence` at top level, `revenue_sources` array ≥2 entries

Key JSON fields (top-level): revenue_confidence (HIGH/MEDIUM/LOW), revenue_sources[], sources_succeeded[], sources_failed[]

Known Issues: - Most of the 6 sources require LLM web browsing — hard to fully script - Revenue confidence calculation is informal (3+ agree = HIGH, etc.) - sources_succeeded/sources_failed tracking is manual

Codification Opportunity: - Confidence scoring → Python rule: count agreeing sources, measure variance - ecdb.com lookup → Python script (known URL pattern) - LinkedIn headcount → Apify script (already used in hiring)

1G: Investor Intelligence (`algolia-intel-investor`)

Purpose: Verbatim executive quotes from earnings calls, SEC 10-K MD&A, Yahoo Finance news. Used heavily by sales-plays skill for exec-language matching.

Execution Model: Python script (collect-investor.py) + LLM enrichment

Aspect	Detail
Script	`collect-investor.py <domain> <output-dir>`
LLM enriches	Earnings call transcript reading, SEC 10-K MD&A and risk factors, media quote extraction via Tavily/WebSearch
Output	`11-investor-intelligence.md` + `11-investor-intelligence.json`
Reads from	`01-company-context.json` (company_name, domain, ticker, public_private)
Gate	File exists, contains verbatim quotes with source citations

Known Issues: - Significant overlap with 1E (financial-public) — both fetch SEC 10-K and earnings calls - For private companies, falls back to CEO/founder interviews — different code path - Quote extraction quality varies wildly between runs

Codification Opportunity: - Share collect-sec-data.py utility with 1E - Standardize quote extraction format with Pydantic model

1H: Hiring Signals (`algolia-intel-hiring`)

Purpose: Identifies ICP-relevant open roles, classifies by buyer tier (Economic Buyer, Technical Buyer, Champion).

Execution Model: Python script (collect-hiring.py) — mostly programmatic

Aspect	Detail
Script	`collect-hiring.py <domain> <output-dir>`
Script does	Apify LinkedIn Jobs scraping, careers page WebFetch, Indeed
LLM enriches	Role classification by tier
Output	`09d-hiring-signals.md` + `09d-hiring-signals.json`
Gate	Both files exist

Known Issues: - Role classification (Economic Buyer / Technical Buyer / Champion) is LLM judgment - Could be 95% keyword-based: "VP" / "Director" / "Head of" → Economic Buyer; "Engineer" / "Developer" → Technical Buyer; "Product Manager" / "Search" in title → Champion

Codification Opportunity: - Role-to-tier mapping → keyword dictionary with LLM fallback for ambiguous titles

1I: Social Signals (`algolia-intel-social`)

Purpose: Scrapes LinkedIn and Twitter/X posts for strategic signals — tech investment, search pain, expansion.

Execution Model: Python script (collect-social.py) + LLM relevance scoring

Aspect	Detail
Script	`collect-social.py <domain> <output-dir>`
Script does	Apify scraping for LinkedIn company posts and Twitter/X posts
LLM enriches	Scores each post for Algolia relevance, extracts strategic signals
Output	`09b-social-signals.md` + `09b-social-signals.json`
Gate	Both files exist, includes "Platform Notes" section

Known Issues: - LLM relevance scoring is expensive and inconsistent across runs - Apify scraping can fail silently (rate limits, auth)

Codification Opportunity: - Relevance scoring → keyword list ("search", "personalization", "digital transformation", "site search") with LLM only for borderline cases

1J: News Signals (`algolia-intel-news`)

Purpose: Last 60 days of news — leadership changes, funding, tech investments, product launches.

Execution Model: Python script (collect-news.py) — mostly programmatic

Aspect	Detail
Script	`collect-news.py <domain> <output-dir>`
Script does	Apify Google News (3 queries), company RSS/newsroom feeds
LLM enriches	News categorization and relevance classification
Output	`09c-news-signals.md` + `09c-news-signals.json`
Gate	Both files exist, `lookback_days` and `collection_date` at top level

Known Issues: - News categorization is LLM-driven but could be rule-based (keyword matching on headlines)

Codification Opportunity: - Category assignment → keyword lists: "CEO" / "appoints" → leadership; "funding" / "raises" → funding; "launches" / "unveils" → product

1K: Partner Intelligence (`algolia-intel-partner`)

Purpose: Maps which Algolia tech partners (Adobe, Salesforce, Shopify, etc.) and SI partners (Accenture, Deloitte, etc.) the prospect uses. Identifies co-sell motion opportunities.

Execution Model: Pure LLM (no Python script)

Aspect	Detail
Script	None
Detection	Crossbeam MCP for overlap, tech stack analysis from 1B, WebSearch for SI relationships
Partner types	Tech Partners (co-sell) + SI/Implementation Partners (relationship/influencer)
Output	`07-partner-intel.md` only — NO JSON
Gate	File exists, ≥ 2000 bytes

Known Issues: - No JSON output — this is a critical gap. Everything downstream that needs partner data must parse prose. - No Python script at all — everything is LLM web searching - Partner detection could use BuiltWith tech stack (already collected in 1B) + a static partner list

Codification Opportunity: - Tech partner detection → Python script cross-referencing 1B tech stack against Algolia partner list - SI detection → WebSearch + structured extraction - Must add JSON output — 07-partner-intel.json

1L: Industry Intel (`algolia-intel-industry`)

Purpose: Vertical benchmarks (Baymard, Forrester, NRF), search conversion stats, trend analysis, expert quotes.

Execution Model: Python script (collect-industry.py) + heavy LLM enrichment

Aspect	Detail
Script	`collect-industry.py <domain> <output-dir>`
Script does	Tavily advanced search for industry reports
LLM enriches	Benchmark context, algolia_angle, competitor search landscape analysis
Output	`06-industry-intel.md` + `06-industry-intel.json`
Gate	Files exist, max 24-month staleness on benchmarks

Known Issues: - Same benchmarks (Baymard, Forrester) are scraped every run — should be a curated dataset - 24-month staleness rule exists but no date-checking logic implemented - LLM synthesis quality varies

Codification Opportunity: - Curate a vertical-benchmarks.json with common stats, refreshed quarterly - Staleness check → Python date comparison - Industry trend templates per vertical → reduce LLM variance

Wave 2 — Query Generation

Query Set (`algolia-intel-queries`)

Purpose: Generates 14-18 typed search queries for the browser audit based on company vertical and product catalog.

Execution Model: Pure LLM

Aspect	Detail
Script	None
Query types	8 types: broad category, specific product, NLP/conversational, typo variants, synonym/colloquial, non-product content, brand, no-results
Output	`05-test-queries.md`
Reads from	`01-company-context.md`, `02-tech-stack.md`
Gate	File exists, > 500 bytes, ≥14 queries across 8 types

Known Issues: - No structured JSON output — queries are in free-form Markdown - Different runs produce different query formats (numbered list vs table vs sections) - No schema enforcing the 8 query types

Codification Opportunity: - Output format → JSON with typed arrays per query type - Some query types can be templated: typo variants (drop letters), brand queries (just use brand name)

Layer 2 — Browser Audit

Browser Testing (`algolia-audit-browser`)

Purpose: Live Playwright-based testing of the prospect's search experience. 20 test steps covering autocomplete, NLP, typos, facets, empty states, merchandising. Produces evidence screenshots.

Execution Model: Playwright + LLM evaluation

Aspect	Detail
Script	`audit-browser.js` (Playwright automation)
Test steps	20 steps (2a through 2t)
Tools	Chrome DevTools MCP or Playwright with stealth mode
Output	`09-browser-findings.md` + `deliverables/screenshots/*.png`
Reads from	`05-test-queries.md`, `02-tech-stack.md`
Gate	≥ 10 screenshots, findings file > 2000 bytes

Key finding fields (from AGENT-CONTEXT.md): id, title, severity, category, tested_query, expected_behavior, actual_behavior, impact_stat, impact_stat_source, screenshot_file, prospect_description, algolia_solution, algolia_case_study_url, algolia_case_study_company, algolia_case_study_result

Known Issues: - Finding schema exists in AGENT-CONTEXT.md but not enforced in skill output - Screenshots need consistent naming convention — currently inconsistent - No JSON output for findings — only Markdown - WAF/bot detection blocks testing on some sites - 20 test steps in one session can hit context limits

Codification Opportunity: - Screenshot naming → {step_id}-{query_slug}.png convention in script - Finding output → structured JSON array matching AGENT-CONTEXT.md fields - WAF bypass strategies → documented per CDN type (from 1B tech stack)

Layer 3 — Synthesis

3A: Business Case (`algolia-synth-business-case`)

Purpose: 6-component ROI model — conversion lift, AOV increase, bounce rate reduction, no-results recovery, speed/latency gain, long-tail discovery. All formulas shown.

Execution Model: LLM (but ROI formulas are deterministic math)

Aspect	Detail
Script	`calculate-roi.py` EXISTS but is NOT called by the skill
Formulas	6 components, each: `impact = baseline × lift_percentage × revenue_attribution`
Output	`{slug}-business-case.md`
Reads from	`08-financial-profile`, `04-competitors`, `09-browser-findings`

Known Issues: - calculate-roi.py already exists but the skill doesn't use it — this is the worst codification miss - All 6 ROI formulas are deterministic given inputs — LLM adds narrative framing but the numbers should come from Python - Labels every input as [FACT] or [ESTIMATE] — good practice but inconsistently applied - No JSON output

Codification Opportunity: - Wire calculate-roi.py into the skill (numbers from Python, narrative from LLM) - Add JSON output with calculated figures + metadata

3B: Sales Plays (`algolia-synth-sales-plays`)

Purpose: AE/BDR playbook — BLUF header, talking points using exec language, SPIN discovery questions, MEDDPICC gap map, objection handling, power map.

Execution Model: Pure LLM — legitimately needs creativity

Aspect	Detail
Script	None needed — this is irreducibly creative
Output	`{slug}-playbook.md`
Reads from	`11-investor-intelligence`, `04-competitors`, `09d-hiring-signals`, `08-financial-profile`, `{slug}-business-case.md`

Known Issues: - Only produces .md — no structured JSON output for the report to consume - Quality depends heavily on investor intel quality (garbage in → garbage out)

Codification Opportunity: - Add JSON output with structured sections (talking_points[], spin_questions[], objections[]) - Template the MEDDPICC structure so only content varies

3C: Report + Deliverables (`algolia-audit-report`)

Purpose: Assembles the final audit-data.json and renders all HTML deliverables (SPA, AE report, battle card, leave-behind, PDF).

Execution Model: LLM reading ALL research files + template rendering

Aspect	Detail
Scripts	`generate-audit-data.py`, `calculate-score.py`, render templates
Output	`{slug}-audit-data.json` + 5 HTML deliverables + PDF
Reads from	*ALL `research/.md` files** + `deliverables/screenshots/`
Template	`audit-data.schema.json` (JSON Schema draft-07) — 29 top-level fields

Known Issues: - Reads .md files, not .json files — this is the context window killer. It ingests 11+ prose documents. - Capability matrix uses fixed row keys (nike_has, asics_has) that must be mapped to actual competitor names — brittle - Should read from .json files exclusively - generate-audit-data.py and calculate-score.py exist but may not be fully wired

Codification Opportunity: - Switch to reading .json files only → massive context reduction - Data assembly → generate-audit-data.py reads all JSON, merges, validates against schema - Score calculation → calculate-score.py (already exists) - LLM only needed for: narrative sections (executive summary, prospect descriptions), case study matching

3D: ABX Campaign (`algolia-campaign-abx`)

Purpose: 5-email sequence, 3 LinkedIn messages, Loom video script, collateral schedule. All personalized from audit findings.

Execution Model: Pure LLM — creative writing

Aspect	Detail
Script	None needed
Output	`abx-campaign/` folder (10 files) + updates `audit-data.json`
Reads from	All research files + `{slug}-audit-data.json`

Known Issues: - Mutates audit-data.json — appends email bodies into a file that's already been rendered and potentially factchecked. Should write to its own abx-data.json. - 10 output files with no JSON schema

Codification Opportunity: - Separate ABX data from main audit-data.json - Email personalization template (structure fixed, content varies) → reduce LLM variance

Layer 4 — Quality

Factcheck (`algolia-audit-factcheck`)

Purpose: 20-dimension quality gate before publish. Verifies stats match, citations are live, quotes are attributed.

Execution Model: LLM evaluation (but many checks are mechanical)

Aspect	Detail
Dimensions	20 across 4 groups: intelligence (1-11), browser (12), synthesis (13-17), deliverables (18-20)
Execution tiers	Quick (~3-5 min), Standard (~15 min), Full (~30-40 min)
Output	`factcheck-report.md`, `correction-manifest.md`, `FACTCHECK_GATE.md`, `skill-feedback.md`
Gate output	PROCEED / WARN / BLOCKED

Known Issues: - 15 of 20 checks are mechanical (file exists? JSON valid? field not null?) — done by LLM unnecessarily - Only 5 checks truly need LLM (narrative quality, quote attribution accuracy, case study relevance) - Evidence tier system (AUTHENTIC > WEBFETCH > WEBSEARCH > NO_SOURCE) is good but not programmatically enforced

Codification Opportunity: - Dimensions 1-12 (file existence + JSON validity) → factcheck-mechanical.py - Dimensions 18-20 (deliverable existence) → factcheck-mechanical.py - Only dims 13-17 (narrative quality checks) → LLM - Evidence tier check → Python regex on [FACT — / [ESTIMATE — labels

Eval (`algolia-audit-eval`)

Purpose: 5-dimension scoring rubric for any skill output — Completeness, Source Density, Instruction Adherence, Data Accuracy, No Fabrication.

Execution Model: LLM evaluation

Aspect	Detail
Scoring	5 dimensions × 2 points each = 10 total. Pass ≥ 7.0
Output	`{skill-name}-eval-report.md`

Known Issues: - No Pydantic for output consistency - Scoring formula could be partially automated (completeness = field count check, source density = label count)

Cross-Module Dependencies

                    ┌──────────────────────────────────────────┐
                    │           Wave 1 (parallel)              │
                    │                                          │
                    │  1A company ──┐                          │
                    │  1B techstack ┤──→ 1D competitors        │
                    │  1C traffic   │                          │
                    │  1E/1F financials                        │
                    │  1G investor                             │
                    │  1H hiring                               │
                    │  1I social                               │
                    │  1J news                                 │
                    │  1K partner                              │
                    │  1L industry                             │
                    └───────────────┬──────────────────────────┘
                                    │
                    ┌───────────────▼──────────────────────────┐
                    │  Wave 2: queries (reads 1A + 1B)         │
                    └───────────────┬──────────────────────────┘
                                    │
                    ┌───────────────▼──────────────────────────┐
                    │  Layer 2: browser (reads queries + 1B)   │
                    └───────────────┬──────────────────────────┘
                                    │
                    ┌───────────────▼──────────────────────────┐
                    │  Layer 3A: business case                 │
                    │  Layer 3B: sales plays                   │
                    │  Layer 3C: report (reads ALL)            │
                    │  Layer 3D: ABX campaign                  │
                    └───────────────┬──────────────────────────┘
                                    │
                    ┌───────────────▼──────────────────────────┐
                    │  Layer 4: factcheck → publish            │
                    └──────────────────────────────────────────┘

Index — project overview
Known-Issues — current failure modes
Refactor-Architecture — planned fixes

Module-Catalog.md

Algolia Search Audit — Module Catalog

Wave 1 — Intelligence Collection (11 modules, parallel)

1A: Company Context (algolia-intel-company)

1B: Tech Stack (algolia-intel-techstack)

1C: Traffic & Engagement (algolia-intel-traffic)

1D: Competitors (algolia-intel-competitors)

1E: Financial Profile — Public (algolia-intel-financial-public)

1F: Financial Profile — Private (algolia-intel-financial-private)

1G: Investor Intelligence (algolia-intel-investor)

1H: Hiring Signals (algolia-intel-hiring)

1I: Social Signals (algolia-intel-social)

1J: News Signals (algolia-intel-news)

1K: Partner Intelligence (algolia-intel-partner)

1L: Industry Intel (algolia-intel-industry)