Final Data Health
Did enriching a fresh copy of production (2.2) actually improve on the current production baseline (2.1)? And what's the final health verdict either way?
Big picture
The enrichment work helped in one specific way and didn't touch most of the rest. The composite scores land within 1 point of each other (2.2: 63.4, 2.1: 62.6, out of 100). That closeness hides a real split: 2.2's entire edge comes from having zero duplicate records, versus 2.1's 28.4% duplication problem. Strip that one dimension out and score content completeness, richness, and distinctness alone, and 2.1, the unenriched baseline, is actually ahead, 63.5 to 59.4.
In plain terms: the enrichment pass didn't make the content itself richer overall, it added a real, valuable new capability (crawled body text) to less than half the records, while several other fields (topic metadata, description/abstract distinctness) came through this copy worse than they started, not better.
Both also share a problem neither copy of production ever had fixed: zero category hierarchy for Documentation, each one's single biggest section. That repeating exactly, at exactly 0%, on both the baseline and the enriched copy, confirms it's a source-pipeline issue from before either copy was taken, not something enrichment could have addressed.
Composite score
A less-than-1-point gap. Read this as "statistically even," not "enrichment won." See the breakdown below for why, and the re-score that isolates content quality from indexing hygiene.
How the score is built, in full
7 dimensions, each scored 0-100 from the exact numbers in 2.1 and 2.2, weighted by how much each one matters for a Content Engagement agent's ability to ground itself in this data. Weights are stated so the score can be argued with, not accepted as a black box.
| Dimension | Weight | 2.2 Enriched | 2.1 Baseline | Ahead |
|---|---|---|---|---|
| Category structure hierarchicalCategories meaningful % |
20% | 41.5 | 42.6 | Baseline |
| Content availability avg of description + abstract meaningful % |
15% | 93.3 | 94.2 | Baseline |
| Content distinctness 100 minus description/abstract identical-copy rate |
10% | 69.1 | 76.2 | Baseline |
| Topic metadata avg of tags/keywords/authors/facets meaningful % |
15% | 17.8 | 28.0 | Baseline |
| Link-health tracking is404 tracked % |
15% | 45.9 | 49.7 | Baseline |
| Deduplication 100 minus duplicate-record % |
15% | 100.0 | 71.6 | Enriched |
| Basic fields avg of title/thumbnail/published_at/lastUpdated meaningful % |
10% | 96.8 | 99.7 | Baseline |
| Weighted composite | 100% | 63.4 | 62.6 | Enriched, by 0.8 points |
The re-score that matters more
The baseline (2.1) wins 6 of the 7 dimensions above. The enriched copy (2.2) only wins on deduplication, but that one dimension is weighted heavily enough (15%) to flip the total. Deduplication is an indexing-hygiene problem, fixable by a script in an afternoon (see 2.1, action item 4). It says nothing about whether enrichment made the content itself better. Re-running the composite without deduplication and link-health tracking (both hygiene/tracking dimensions, not content-quality dimensions) isolates the real question this comparison is actually about:
On content quality alone, the unenriched baseline is ahead by 4.1 points. This matches what 2.2 found directly: its topic-metadata fields (tags, keywords, authors, facets) are all thinner than 2.1's equivalent fields, and its description/abstract duplication rate is worse. The enrichment's one clear asset, the partial body-enrichment layer (40.6% of records), isn't part of either score because 2.1 has no equivalent field to compare against, not because it doesn't count. It's real value, just not something this particular scoring method can credit properly.
Enrichment added one real, valuable capability (crawled body text) to less than half the records, fixed indexing duplication cleanly, and left almost every other data-health dimension no better, and in some cases worse, than where it started. Neither copy of the data is demo-ready as-is. The fastest path to a genuinely better dataset is not re-running more enrichment on the AC2 copy, it's finishing the two low-effort, high-impact fixes that apply to both copies identically: the link-health sweep and the Documentation category backfill (both detailed in 2.1 and 2.2's action tables). Those two fixes move the needle regardless of which copy the POC ends up querying.
What this implies for the decisions ahead
- If the POC queries the enriched copy (consistent with the Chapter 1 decision that Content Engagement is the AC2 work), the honest starting point is that its content is currently thinner than the unenriched baseline, not richer. The zero-duplication result is real credit, but it doesn't close that gap on its own.
- Finishing the enrichment pass (currently 40.6% coverage) is probably higher-value than matching the baseline's tag/keyword fill rates field-for-field, since crawled body text is the one thing this copy has that the baseline doesn't.
- The shared Documentation category gap should be escalated once, to whoever owns the source content pipeline, rather than fixed twice independently on each copy.
- Whatever process produced the enriched copy's zero-duplication result should be understood and, if possible, applied back to fix the baseline's 28.4% duplication problem. That's real, transferable process knowledge, not a one-off.