Competitive Intelligence
logs/2026-06-27-source-reliability-pass.md
Source Reliability Pass - 2026-06-27
Goal
Make quiet days more trustworthy by replacing stale/blocked direct sources and disabling sources that should be manual-only or search-backed.
Changes Made
| Source | Old state | Change |
|---|---|---|
| Elastic release notes | 404 | Replaced with https://www.elastic.co/docs/release-notes/elasticsearch |
| Lucidworks blog | 403 | Replaced with Lucidworks docs latest-changes page |
| Lucidworks product | 403 | Replaced with Lucidworks platform docs page |
| Yext search product | 503 | Replaced with https://www.yext.com/platform/search |
| Algonomy blog | 404 | Replaced with https://algonomy.com/blogs/ |
| MCP blog | 404 | Replaced with https://blog.modelcontextprotocol.io/ |
| OpenAI blog | 403 | Replaced with official RSS feed https://openai.com/news/rss.xml |
| Gartner MQ document | 403/gated | Disabled in direct collector; manual-only until public summary/fallback exists |
| Perplexity blog | 403 | Disabled in direct collector; tracked through search queries for now |
| Reddit ecommerce | 403 | Replaced with old Reddit RSS endpoint |
| Reddit SEO | 403/429 | Disabled until stable API/search monitor exists |
| Reddit MachineLearning | 403/429 | Disabled until stable API/search monitor exists |
Verification
VPS direct collection dry run:
Sources loaded: 43
snapshots=39 signals=10 errors=1
VPS source coverage audit:
enabled=39 checked=39 ok=38 failed=1 missing=0 snapshots=39 signal_sources=10 changed_sources=2
Remaining source needing attention:
Martech Edge [error] https://martechedge.com - The read operation timed out
Daily synthesis after repair:
Synthesis packet: 2 signals, 10 ledger sources
Markdown saved: .../briefs/2026-06-27.md
HTML saved: .../reports/2026-06-27.html
Quality score: 1.00
Dashboard:
Dashboard saved: .../artifacts/competitive-research/dashboard/index.html
Interpretation
This is a real reliability improvement: failed direct sources dropped from 12 to 1, and enabled source coverage improved to 38/39 successful.
However, this does not yet make the intelligence excellent. The current daily report can still say "source changed" without enough semantic explanation of what changed. The next quality step is to improve delta extraction and weekly gates so changed-page signals become analyst-grade intelligence.
Next
- Add a timeout/fallback rule for Martech Edge.
- Add a weekly baseline-only gate.
- Improve changed-source summaries so they explain what changed, not merely that the page changed.