Dashboard/Scout/wiki/syntheses/current-status-2026-05-22.md
Scout

wiki/syntheses/current-status-2026-05-22.md

Scout Current Status

Last Updated: 2026-05-22

Executive Summary

Scout is now an app-first, provider-aware web intelligence platform foundation. It can run from the local HTTP app, preserve run artifacts, validate UI workflows through browser automation, and execute a live target matrix. The latest build added a real Scout Browser workbench backed by Playwright capture.

Scout is not yet finished as a production-grade product extraction system. It now truthfully reports blocked hard-site behavior and preserves evidence, but it still needs User Browser mode and browser-DOM product parsing to solve sites that are visible in the user's browser but blocked in Scout's isolated session.

Built So Far

Platform

  • Standalone package foundation.
  • CLI and HTTP run surfaces.
  • Skill documentation direction.
  • Execution modes: auto, crawl4ai, webfetch, websearch, browser/scout-browser, saved, api.
  • Standard artifact contract: manifest.json, records.json, records.jsonl, source_pages.json, blocked_pages.json, validation.json, extraction_report.md.
  • Citation-grade source evidence model with source registry and record-level citations.

Use Cases

  • Product intelligence foundation.
  • Company intelligence foundation.
  • PRISM bundle foundation.
  • Investor intelligence foundation.
  • Careers/hiring foundation.
  • Jobs URL-seeded V1 with scoring/profile support.
  • News/blog foundation.
  • Research/docs/website-quality scaffolds.

App UI

  • App-first shell with Run Setup, Live Execution, Results Review.
  • Working directory default set to /Users/arijitchowdhury/AI-Development/Scout/tests.
  • Native macOS folder picker endpoint.
  • Crawl Settings chips and Developer Details accordion.
  • Navigation screens for History, Presets, Targets, Data, Integrations, Settings, and Help.
  • Active Run banner so navigation does not lose the running workflow.
  • Browser Workbench with screenshot-capable evidence rendering.

Browser Workbench

  • scout-browser mode launches Playwright.
  • Captures screenshot, DOM, rendered text, links, console errors, network failures, status code, and viewport metadata.
  • Writes browser artifacts under the selected output directory.
  • Marks access-denied/blocked captures as blocked evidence rather than success.
  • Represents user-browser honestly as bridge-required.

Verification Status

Latest verified commands:

Command Result
python3 -m pytest tests/e2e/test_app_ui_exhaustive.py -q 14 passed
SCOUT_LIVE_TESTS=1 python3 -m pytest tests/live/test_app_live_targets.py -q 39 passed
python3 -m pytest tests/unit/ -q 196 passed
python3 -m pytest tests/ -q 216 passed, 41 skipped
python3 -m pyright scout/ 0 errors
ruff check scout/ tests/ && ruff format --check scout/ tests/ passed

What Is Still Pending

High Priority

  1. User Browser mode - Chrome CDP or browser-extension bridge. - Explicit consent/security model. - Capture from the user's actual visible browser tab.

  2. Browser-DOM product parser - Parse visible listing cards from captured DOM/text. - Produce listing-level product.v1 records even when detail pages are blocked. - Add completeness and citation coverage for browser-derived records.

  3. Workbench controls - Enable Capture, Extract, Save Evidence, Refresh, Back, and Forward through a real browser session API.

  4. Product Workbench quality - Better Algolia-ready record preview. - Missing-field display. - Image/price/review/color/SKU enrichment where available.

Medium Priority

  1. Deepen semantic extraction for company, PRISM, investor, careers, and news.
  2. Build provider adapters for ATS/job boards and official APIs.
  3. Improve live test quality assertions beyond record/source counts.
  4. Expand workflow-specific mockups for non-product use cases.
  5. Cleanly separate public fixtures from private profiles and vault-only data.

Next Step

The next logical build slice is User Browser Capture V1:

  • Add a Settings screen section for browser bridge status.
  • Implement a CDP connection option for Chrome launched with remote debugging.
  • Add /app/browser/connect, /app/browser/tabs, and /app/runs/{run_id}/capture-user-browser style endpoints.
  • Capture active-tab screenshot/DOM/text from the user-approved session.
  • Feed captured evidence into the existing source/blocked/artifact model.
  • Add E2E tests for bridge-not-connected and mocked connected states.

This is the path that can make “my browser can view Estée Lauder, so Scout should capture it” real.