ADR: Citation-Grade Source Evidence

Date: 2026-05-20 Status: Accepted

Decision

Scout records must support citation-grade provenance. Page-level provenance in source_pages.json is necessary but insufficient for PRISM-quality research and Algolia-ready evidence. Each record should carry structured citations[] that identify the source page and the field or claim supported by that source.

Implementation

• SourceEvidence now has a deterministic source_id derived from provider and URL identity.
• Citation captures source_id, source_url, field, claim, snippet, optional selector, and confidence.
• source_pages.json is now a source registry with URL, provider, fetch state, blocked/error state, content hashes, title, and content availability flags.
• records.json and records.jsonl carry record-level citations[].
• validation.json includes missing_citations warnings when records lack structured citations.
• extraction_report.md includes source and citation coverage counts.

Consequences

• Existing source_url and source_urls fields remain for compatibility.
• New vertical processors should not be considered production-ready unless citation coverage is visible.
• Browser, WebFetch, Crawl4AI, API, and saved evidence can all normalize into the same citation model.