Competitive Intelligence
logs/2026-06-26-ci-system-audit.md
CI System Audit - 2026-06-26
Executive Finding
The CI engine is now operationally stable enough to continue, but the intelligence quality is still yellow. The current ledger proves collection and artifact generation, not high-quality competitive judgment.
The most important change from today's fix: baseline captures are no longer delivered as daily competitive moves. That removes the most dangerous current failure mode: forced narrative from first-run or unchanged source captures.
Live Coverage Snapshot
Window: 2026-06-26 to 2026-06-26.
| Metric | Value |
|---|---|
| Enabled sources | 43 |
| Checked sources | 43 |
| Successful sources | 31 |
| Failed sources | 12 |
| Missing sources | 0 |
| Snapshots in window | 43 |
| Signals in window | 27 |
| Baseline signal sources | 27 |
| Changed signal sources | 0 |
| Material signal sources | 0 |
Interpretation: collection ran broadly, but every stored signal in the window was baseline, not a material delta. The correct daily delivery is a quiet-day brief with a source-health warning.
Source Health Classification
| Source | Type | Status | Classification | Evidence / repair path |
|---|---|---|---|---|
| Algolia MCP guide | docs | ok | healthy baseline | HTTP 200, baseline only |
| Algolia public MCP guide | docs | ok | healthy baseline | HTTP 200, baseline only |
| Bloomreach blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Bloomreach product updates | changelog | ok | healthy baseline | HTTP 200, baseline only |
| Bloomreach roadmap | roadmap | ok | healthy baseline | HTTP 200, baseline only |
| Bloomreach customers | case study | ok | healthy baseline | HTTP 200, baseline only |
| Constructor blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Constructor product | product | ok | healthy baseline | HTTP 200, baseline only |
| Constructor customers | case study | ok | healthy baseline | HTTP 200, baseline only |
| Coveo blog | blog | ok | healthy, low extraction confidence | HTTP 202, no signal stored |
| Coveo press releases | press | ok | healthy baseline | HTTP 200, baseline only |
| Coveo platform | product | ok | healthy, low extraction confidence | HTTP 202, no signal stored |
| Coveo commerce | product | ok | healthy, low extraction confidence | HTTP 202, no signal stored |
| Coveo case studies | case study | ok | healthy, low extraction confidence | HTTP 202, no signal stored |
| Google Cloud AI blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Google Vertex AI page | product | ok | healthy baseline | HTTP 200, baseline only |
| Google GenAI app builder docs | docs | ok | healthy baseline | HTTP 200, baseline only |
| Elastic blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Elastic enterprise search | product | ok | healthy baseline | HTTP 200, baseline only |
| Elastic release notes | changelog | error | stale URL | HTTP 404, replace URL |
| Lucidworks blog | blog | error | blocked | HTTP 403, replace with RSS/search fallback |
| Lucidworks products | product | error | blocked | HTTP 403, replace with alternate crawl/search fallback |
| Yext blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Yext search product | product | error | unstable/blocked | HTTP 503, retry and find alternate canonical page |
| Algonomy blog | blog | error | stale URL | HTTP 404, replace URL |
| Meilisearch blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Meilisearch GitHub releases | changelog | ok | healthy baseline | HTTP 200, baseline only |
| Typesense blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Typesense GitHub releases | changelog | ok | healthy baseline | HTTP 200, baseline only |
| AWS big data blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Perplexity blog | blog | error | blocked | HTTP 403, replace with RSS/news/search fallback |
| OpenAI blog | blog | error | blocked | HTTP 403, use RSS/news/search fallback |
| MCP blog | protocol | error | stale URL | HTTP 404, replace with current docs/news source |
| Google A2A releases | protocol | ok | healthy baseline | HTTP 200, baseline only |
| Glean blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Gartner MQ document | analyst | error | gated/blocked | HTTP 403, mark manual-only or use public summary |
| CMSWire Digital Experience | news | ok | healthy baseline | HTTP 200, baseline only |
| Martech Edge | news | ok | healthy baseline | HTTP 200, baseline only |
| Contra Collective blog | blog | ok | healthy baseline | HTTP 200, baseline only |
| Reddit ecommerce | forum | error | blocked | HTTP 403, replace with API/search/manual monitor |
| Reddit SEO | forum | error | blocked | HTTP 403, replace with API/search/manual monitor |
| Reddit MachineLearning | forum | error | blocked | HTTP 403, replace with API/search/manual monitor |
| Hacker News | forum | ok | healthy baseline | HTTP 200, baseline only |
Quality Scores
| Artifact | Score | Rationale |
|---|---|---|
| Daily pulse before baseline filter | 0.35 | It turned baseline captures into action recommendations, creating false urgency. |
| Daily pulse after baseline filter | 1.00 | Correctly reports no material signal and surfaces failed-source caveat. |
| Weekly digest after renderer repair | 0.75 | Artifacts generate reliably, but weekly narrative still needs stronger baseline-vs-delta distinction. |
| Source registry | 0.72 | Broad enough for pilot, but 12 failed sources and several strategic blind spots remain. |
| Ledger schema | 0.68 | Captures sources, snapshots, signals, and synthesis runs, but lacks first-class report index, source health events, action items, bot deliveries, analyst review state, and stakeholder readiness flags. |
Failure Modes Found
| Failure mode | Evidence | Status |
|---|---|---|
Weekly renderer NameError |
Weekly path called undefined render_weekly_html_report |
Fixed and tested |
| Runtime ledger deletion by broad vault sync | VPS ci.sqlite was missing after project mirror sync |
Guard added; ledger recreated |
| Baseline capture delivered as daily move | Constructor customer baseline appeared as action-worthy item | Fixed and tested |
| Quiet-day overconfidence | 12 failed sources can hide missed deltas | Warns in daily output; source repair still needed |
| Blocked direct sources | Gartner, Lucidworks, Perplexity, OpenAI, Reddit return 403 | Needs fallback strategy |
| Stale source URLs | Elastic release notes, MCP blog, Algonomy blog return 404 | Needs source repair |
| Weekly baseline/delta ambiguity | Weekly window includes 27 baseline signals and 0 changed signals | Needs stronger weekly gate |
| Local-vs-VPS dependency drift | VPS container lacks pytest; venv has runtime deps | Use local tests plus VPS script smoke tests |
Prioritized Improvements
- Keep weekly renderer and baseline filter as mandatory regression tests.
- Add report index and action queue tables before dashboard build.
- Repair or reclassify blocked/stale sources.
- Add weekly quality gate: no battlecard/action recommendation from baseline-only signals.
- Add source health view before any stakeholder-facing dashboard mode.
- Add analyst review status before any Algolia CMO or external stakeholder access.