Dashboard/Scout (Second-Brain)
Project

Scout (Second-Brain)

Vault: second_brain   Path: Scout

index.md

Scout — Project Index

Current state (compiled truth — 2026-07-07)

Scout SHIPPED to production for the Monday beta launch (2026-07-06): full site rebuild live (mint-neumorphism redesign, new pricing, anonymous demo API), 888 tests green, live-verified; prod-launch hardening complete (Stripe checkout, usage metering, backups). This supersedes the 2026-06-29 "public launch blocked" state below — that section is retained as history. Source: tracker log 2026-07-06.

Prior state (2026-06-29, superseded):

What: Scout is an evidence-first web acquisition and record-preparation engine built on Crawl4AI. It runs locally, through Docker, through a hosted private-beta API, as a CLI, as a FastAPI HTTP service, and as a Claude/Codex skill backend.

Why: Scout gives downstream agents and workflows a trusted acquisition layer: fetch/crawl/render pages, preserve source evidence, structure records, explain blocked cases, and export artifacts. Consumers such as PRISM, Competitive Intelligence, Algolia search-audit skills, job workflows, and product/catalog workflows use Scout for acquisition and evidence; they own interpretation.

Repo: ~/Dropbox/AI-Development/Scout/ — branch codex/scout-platform-foundation

Status (2026-06-29): Controlled private beta is credible with limits. Public launch is blocked until founder/legal/risk decisions close. Latest pushed Scout commit observed in this checkpoint: 8aa8b09 Add beta developer guide page.

Launch truth: Private beta = ready_with_limits; public launch = blocked; Codex-actionable public launch blockers = 0 until Arijit/legal/payment/risk decisions are made.

Pricing truth: Arbitrary $22 one-time and $9/month hosted concepts are rejected as placeholders. Hosted pricing must be derived from unit economics: fixed monthly costs, variable usage costs, payment fees, support/security/maintenance allocation, target margin, and break-even volume.

Website polish truth: The launch website now uses a Scout SVG mark and wordmark, tighter homepage density, and scroll-aware section navigation.

Key Artifacts

Artifact Path
Resume state repo: SESSION.md
Launch evidence index repo: docs/product/launch-evidence-index-2026-06-29.md
Release checklist repo: docs/product/release-checklist.md
Launch decision dashboard repo: docs/product/launch-decision-dashboard-2026-06-29.md
Public launch action packet repo: docs/product/public-launch-action-packet-2026-06-29.md
Private beta invitation packet repo: docs/product/private-beta-invitation-packet.md
Private beta tester handoff repo: docs/product/private-beta-tester-handoff.md
Website route verification repo: docs/product/website-route-render-verification-2026-06-29.md
Product feature list repo: docs/product/feature-list.md
Differentiation repo: docs/product/differentiation.md
Unit economics and pricing repo: docs/product/unit-economics-and-pricing-model-2026-06-29.md; wiki/syntheses/unit-economics-pricing-2026-06-29.md
Competitor website knowledge repo: docs/competetor-website-knowledge/
Pydantic contracts repo: scout/core/types.py
Launch website repo: website/
Architecture decisions This wiki: wiki/decisions/
Project relationships wiki/syntheses/downstream-consumers-and-project-relationships.md

API Surface

Endpoint Function Status
POST /scrape URL → markdown Working
POST /extract URL + schema → structured records (LLM) Working
POST /crawl Recursive BFS site crawl Working
POST /products Product catalog crawl → Algolia/export-ready records Working for beta-supported cases
POST /structure Raw HTML → markdown + records (raw://) Working
POST /harvest CDP URL → records from live cleared tab Working
POST /map URL discovery/sitemap Working
POST /screenshot Full-page screenshot Working
POST /run/{use_case} High-level use-case runs Beta; use certification docs before claiming per-vertical production readiness
POST /v1/hosted/scrape Hosted private-beta scrape with credits Verified for controlled beta
POST /v1/billing/stripe/checkout-session Hosted beta checkout session Deterministic coverage; real Stripe smoke still open
Public website routes /, /quickstart, /guide, /examples, /pricing, /status, /beta, /legal, /terms, /privacy Verified locally through scout serve

Architecture Invariants

  1. Crawl4AI is THE core — for fetch AND extract. No content path bypasses it. Screencast viewport exempt.
  2. Scope boundary — Scout = acquire + extract. PRISM/skills = interpret. Scout never has an opinion.
  3. Universal engine — consumers import Scout; scout.core never imports a consumer.
  4. Human-assisted rung — for behavioral walls (PerimeterX, DataDome), human must solve. Embedded canvas cannot guarantee bypass.
  5. Pydantic everywhere — no raw dicts crossing boundaries.
  6. Launch claims are evidence-bound — if a claim is not in the launch evidence index or validation report, treat it as unverified.

Downstream Consumers

Consumer / Project How it uses Scout Scout feature used
PRISM / Chowmes-PRISM Acquisition layer for prospect intelligence before the Hermes sales coach interprets findings. /scrape, /structure, /harvest, company/news/careers/investor records, source evidence
Competitive Intelligence Repeatable acquisition primitive for CI source collection and benchmarked scrape quality. /scrape, quality fields, raw/clean markdown, source artifacts
Algolia Search Audit skills Evidence gathering for company, industry, competitor, hiring, investor, news, social, and tech-stack modules. HTTP API, skill backend, Crawl4AI acquisition, citations
Product/catalog workflows Prepare product records for Algolia and other destinations. /products, product exports, Algolia preview/push, JSON/JSONL/CSV/SQLite/Sheets bundle
Job hunter workflows Capture careers pages and ATS/job pages for later matching and scoring. careers/jobs verticals, source evidence, run artifacts
  • overview — architecture and component map
  • requirements — PRD summary + acceptance criteria status
  • ADRs (one per decision): 0001-scope-boundary · 0002-universal-acquisition-engine · 0003-crawl4ai-core-law · 0004-human-assisted-acquisition · 2026-06-29-private-beta-launch-surface
  • dev-log — append-only build log
  • open-questions — unresolved items
  • downstream-consumers-and-project-relationships — who uses Scout and why

Folder catalog (auto-added 2026-07-07 clean-brain — every note must be reachable from its index)

All files (15)