wiki/decisions/2026-06-30-wave2-similarweb-screenshots.md
ADR: Wave-2 traffic comes from manual SimilarWeb screenshots
Date: 2026-06-30 Status: Accepted
Context
Wave-2 of the report standardization is 7 dataless companies (dell, footlocker, jbl, michaelkors, thenorthface, torrid, autozone) that need full re-audits. The traffic module needs SimilarWeb, but the SimilarWeb API key was revoked (401) and the browser-scraper session was expired. This was the one hard blocker stopping Wave-2.
Decision
Use manually captured, logged-in SimilarWeb screenshots as the traffic source for Wave-2. The user
captured all 7 companies (10 tabs each: overview, performance, technologies, competitors, traffic and
engagement, marketing channels, geography, demographics, audience interests, subdomains) at
PIP/docs/temp/similarweb-wave2/. Traffic data is extracted from these screenshots by vision rather
than from the dead API.
Rationale
The data exists and is real (verified, logged-in, no paywall blur). Reading it from screenshots removes the only dependency that needed a paid or live key. With this, every Wave-2 module is keyless: Scout, Gemini-grounded, detect-search on the VPS, yfinance, and these screenshots.
Alternatives Considered
| Option | Why rejected |
|---|---|
| Rotate the SimilarWeb key and re-run the API path | Costs a paid key; the screenshots already exist |
| Re-auth the browser scraper | Same captured data, but the screenshots are already in hand |
| Leave Wave-2 blocked | No longer necessary; the blocker is gone |
Consequences
- The pipeline's
collect-traffic.pyexpects the SimilarWeb API, so Wave-2 needs a small vision-extraction step (screenshots to03-traffic-data.json) instead of the API call. - Wave-2 is now fully runnable and keyless. The earlier ledger (
spike-unify-audit/run/FINAL-REPORT.md) that listed SimilarWeb as the blocker is superseded by this. - Corrects memory
feedback-wave2-blocker-similarweb-mcp(now marked resolved). - Related: scout, detect-search, right-tool-for-each-job.