wiki/syntheses/data-health-final-verdict.md
Data health — final verdict
Did enriching a fresh copy of production (data-health-enriched-copy) actually improve on the current production baseline (data-health-current-production)? Composite score built from 7 weighted dimensions, computed 2026-07-12.
Composite score
| Score | |
|---|---|
| Enriched copy (AC2) | 63.4 / 100 |
| Baseline (current production) | 62.6 / 100 |
Less than 1 point apart — read as statistically even, not "enrichment won."
The score, in full
| Dimension | Weight | Enriched | Baseline | Ahead |
|---|---|---|---|---|
| Category structure (hierarchicalCategories meaningful %) | 20% | 41.5 | 42.6 | Baseline |
| Content availability (desc+abstract meaningful %) | 15% | 93.3 | 94.2 | Baseline |
| Content distinctness (100 − desc/abstract identical %) | 10% | 69.1 | 76.2 | Baseline |
| Topic metadata (tags/keywords/authors/facets avg) | 15% | 17.8 | 28.0 | Baseline |
| Link-health tracking (is404 tracked %) | 15% | 45.9 | 49.7 | Baseline |
| Deduplication (100 − duplicate %) | 15% | 100.0 | 71.6 | Enriched |
| Basic fields (title/thumbnail/published_at/lastUpdated avg) | 10% | 96.8 | 99.7 | Baseline |
| Weighted composite | 100% | 63.4 | 62.6 | Enriched, by 0.8 |
The re-score that matters more
The baseline wins 6 of 7 dimensions. The enriched copy only wins on deduplication — an indexing-hygiene fix, not a content-quality one — but it's weighted heavily enough (15%) to flip the total. Re-scoring without deduplication and link-health tracking (the two hygiene/tracking dimensions, not content-quality ones) isolates the real question:
| Content-only score | |
|---|---|
| Enriched copy | 59.4 |
| Baseline | 63.5 |
On content quality alone, the unenriched baseline is ahead by 4.1 points. The enrichment's real asset — partial body-text crawl coverage — isn't part of either score because the baseline has no equivalent field to compare against. Real value, just not credited by this scoring method.
Final verdict
Enrichment added one real, valuable capability (crawled body text) to less than half the records, fixed indexing duplication cleanly, and left almost every other data-health dimension no better — in some cases worse — than where it started. Neither copy is demo-ready as-is. The fastest path to a genuinely better dataset isn't more enrichment on the AC2 copy — it's the two low-effort, high-impact fixes that apply identically to both copies: the link-health sweep and the Documentation category backfill. Those move the needle regardless of which copy the POC ends up querying.
What this implies (per 2026-07-12-content-engagement-motion-decisions)
Since Content Engagement is the AC2 work (decision 5), the honest starting point is that AC2's content is currently thinner than the unenriched baseline, not richer. Priorities:
- Finish the enrichment pass (currently 40.6% coverage) — probably higher-value than matching the baseline's tag/keyword fill rates field-for-field, since crawled body text is AC2's one real differentiator.
- Escalate the shared Documentation category gap once, to whoever owns the source content pipeline, rather than fix it twice independently.
- Understand and, if possible, replicate whatever process gave the enriched copy its zero-duplication result, back onto the baseline's pipeline.