ADR-0003: Crawl4AI Core Law

Date: 2026-06-21 Status: LOCKED Decision: Crawl4AI is THE engine — for fetch AND extract. No content path may bypass it. Screencast viewport exempt.

Context

Arijit stated this twice as a hard rule. Audit found 3 raw-Playwright paths that bypassed Crawl4AI for content acquisition. The raw paths produced giant text blobs instead of structured markdown.

Decision

Every content acquisition path routes through Crawl4AI: - CDP-attach (acquire_open_page): Crawl4AI attaches to an already-open browser via CDP, reads the tab without re-navigating (js_only=True). - raw:// scheme (structure_capture): Crawl4AI processes held HTML bytes without network fetch. - Standard scrape: Crawl4AI fetches and processes the URL.

The embedded screencast viewport (live_browser.py) is exempt — it's a viewing window, not content acquisition.

Consequences

• No page.content() → inner_text() paths for content (that's bypassing the engine).
• A giant raw blob beside a working extractor = a bypassed path, not a missing feature.
• raw:// + html → crawler.arun() processes bytes in place, status 200, no fetch.