ADR: Daily delta-sync from Visibility to the four indices

Date: 2026-06-18 Status: Accepted

Context

The live Visibility ALGOLIA_WWW_PROD_V2 (1QDAWL72TQ) is the upstream source of truth — new blog posts, releases, and website updates land there first. Our four FLAGSHIP indices are seeded from a one-time copy, which will go stale.

Decision

Build an incremental, data-only sync that periodically (≈ end of day) pushes deltas (new/changed records) from the Visibility index into all four FLAGSHIP indices. The sync: - Updates records only — it never modifies index configuration. Each of the four indices keeps its own settings/mode (they intentionally differ). - Applies the same intake filters as the seed: English only, deduped by url. - Treats Visibility as read-only upstream; writes only to our FLAGSHIP indices.

Rationale

• Keeps the demo current without manual re-seeding.
• Separating data from config is essential — the whole 2×2 depends on each index keeping its distinct config while sharing identical data.

Alternatives Considered

Consequences

• New pipeline component: change detection (objectID + lastUpdated/content hash), delta upsert into 4 indices, scheduling (cron/end-of-day).
• Must verify post-sync that each index's config is untouched (mode/settings preserved).
• Open mechanics tracked in open-questions.