Dashboard/Competitive Intelligence/ci-command-center-project-map.md
Competitive Intelligence

ci-command-center-project-map.md

CI Command Center Project Map

Last updated: 2026-06-27

One-Sentence Intention

Build Competitive Intelligence into a standalone Chowmes product that gives Arijit and, later, selected Algolia stakeholders source-backed competitive signals, weekly patterns, action recommendations, source health, archive access, and a dedicated CI bot, with Athena supervising quality rather than delivering daily reports.

Current Truth

We are not done building the CI product, and the roadmap has been corrected.

The first real phase is now Market Truth and Differentiation, not product build.

We have finished the first foundation layer:

  • Weekly CI no longer crashes.
  • Daily CI no longer treats baseline captures as fresh competitor moves.
  • The live CI ledger is protected from vault sync deletion.
  • The first report/archive/action/source-health data model exists.
  • A first static dashboard artifact exists.
  • The CI workspace has current audit, research, product spec, and ADR notes.
  • The first source reliability pass reduced failed direct sources from 12 to 1.
  • The evidence basis and market benchmark have been made explicit, and the plan should now be judged against that benchmark rather than generic dashboard assumptions.
  • Scout has been tested against direct HTTP on representative CI sources and is now promoted to a first-class collector path, not merely a fallback.

That means the project has a prototype foundation, but product build should continue only where it supports the differentiation thesis.

The accepted decision is: build a narrow Algolia-specific decision layer, not a generic CI platform.

What Exists Now

Layer Exists? Current state
Source registry Yes 43 configured rows, 39 enabled after disabling direct-unfriendly sources
SQLite ledger Yes Live on VPS at artifacts/competitive-research/ci.sqlite
Daily report Yes Baseline-aware, quiet-day honest, still source-quality limited
Weekly report Yes Markdown, HTML, and PDF generate
Report archive model Yes report_index table exists
Source health model Partial Health can be computed; latest audit shows 38/39 enabled sources successful
Action queue model Yes action_items table exists and can persist recommendations
Bot delivery model Schema only bot_deliveries exists, no dedicated CI bot yet
Dashboard Prototype Static artifact with Today, Weekly, Sources, Signals, Actions, Archive
Scout collector Started Smoke test completed; Scout won on Google AI blog and Constructor product, direct HTTP won on some simple/static sources; separate /ci/scrape route was rejected and Scout CI acquisition must use the existing /scrape architecture
Athena oversight Designed Not fully automated yet
Stakeholder mode No Explicitly out of v1 until auth and claim-quality controls exist
MyOS Command Center No Deferred until CI is reliable

What Was Actually Fixed

1. Weekly Report Failure

Problem:

The weekly path called a weekly HTML renderer that did not exist.

Effect of fix:

Weekly CI can now produce decision artifacts instead of failing before the scheduled run.

Why it matters:

Weekly pattern synthesis is one of the core CI rituals. A Command Center cannot exist if its weekly intelligence artifact is unreliable.

2. Baseline-vs-Delta Confusion

Problem:

The daily run could treat a first-time source capture as a competitive move.

Effect of fix:

Baseline captures are filtered out of daily material signals.

Why it matters:

This prevents fake urgency. CI must be trusted enough to say "nothing material happened" instead of manufacturing action.

3. Runtime Artifact Deletion Risk

Problem:

Vault sync could delete runtime-only artifacts such as the ledger, raw captures, reports, PDFs, and dashboard.

Effect of fix:

Sync now preserves the full CI runtime artifact directory.

Why it matters:

The dashboard, bot, and action workflow all depend on a persistent historical ledger. Losing the ledger means losing institutional memory.

4. Dashboard/Data Foundation

Problem:

There was no structured way to power report archive, source health, action items, or bot delivery tracking.

Effect of fix:

The ledger now has tables for report index, source health events, action items, and bot deliveries.

Why it matters:

The dashboard can now be built from operational records instead of scraping random files.

5. Dashboard Prototype

Problem:

There was no CI dashboard surface.

Effect of fix:

The first static dashboard artifact exists with six v1 views.

Why it matters:

This proves the product shape and gives us a concrete surface to improve, but it is not the final private app yet.

What Is Still Broken Or Not Good Enough

Problem Why it matters Next fix
Collector strategy is not source-specific Direct HTTP and Scout each win on different source types Add collector router with source-level strategy
Scout output includes consent boilerplate on some pages Pollutes semantic delta extraction Add Scout cleanup/quality scoring
RSS/feed sources are treated like generic pages Creates oversized noisy captures Add feed parser path
1 enabled source currently fails by timeout Quiet days are much more trustworthy, but not perfect Add timeout/fallback for Martech Edge
Weekly can still include baseline-only signals Weekly narrative can become forced Weekly quality gate
Static dashboard is not served as an app It is not yet a convenient command surface Private dashboard serving
CI bot does not exist Athena still owns delivery by default Dedicated bot after token approval
Source health events are not populated over time We can see current health, but not enough trend history Write health events during runs
Action queue is primitive Actions exist but are not yet assigned/routed/reviewed deeply Add action workflow commands/API
Research is a benchmark note, not a full dossier Product bar is not yet deep enough Deep CI platform/workflow research
OpenRouter auxiliary model warning exists Model-backed synthesis may fail/degrade Separate billing/model-routing investigation

Corrected Roadmap Sequence

Build 0: Market Truth And Differentiation

Goal:

Understand what paid CI systems already do and decide why Chowmes CI should exist.

Status:

Completed as first pass on 2026-06-27.

Artifacts:

  • research/2026-06-27-ci-market-truth-holistic-report.md
  • research/2026-06-27-research-brief.md
  • research/vendor-benchmark/
  • research/ci-tool-feature-matrix.md
  • research/ci-market-patterns.md
  • research/algolia-ci-relevance-map.md
  • research/chowmes-ci-differentiation-thesis.md
  • decisions/2026-06-27-build-or-not-build-ci.md

Decision:

Build only the narrow Algolia-specific decision layer. Do not build a generic CI platform.

Build 1: Data Acquisition And Collector Router

Goal:

Make the data supply chain trustworthy and source-specific.

Status:

Started. Direct source reliability improved, and Scout-vs-direct smoke testing has been completed for representative sources.

Tasks:

  1. Replace stale 404 URLs. Done.
  2. Reclassify gated analyst sources as manual-only. Done for Gartner.
  3. Add fallback sources for blocked 403 sources. Done for Lucidworks, OpenAI, Reddit ecommerce. Perplexity remains search-backed only.
  4. Add source-level collector strategy: direct_http, scout_scrape, rss_feed, parallel_search, parallel_monitor, or manual_only. Started in the CI runner.
  5. Integrate Scout as a secure first-class collector using X-API-Key. Started; CI runner can route scout_scrape through Scout's existing /scrape endpoint.
  6. Add RSS/feed parsing so feeds do not become oversized generic scrape captures. Pending; Scout now detects feed URLs and recommends rss_feed, but the CI feed parser still needs implementation.
  7. Add source repair recommendations to the dashboard/action queue.
  8. Verify source coverage improves from 31/43 successful. Done: 38/39 enabled successful for direct collection.

Done when:

  • Every source is classified as healthy, fallback-backed, manual-only, or intentionally disabled.
  • Quiet-day reports show honest confidence based on source health.
  • Every source has an explicit collector strategy and fallback strategy.
  • Scout, direct HTTP, and RSS/feed captures record method-level quality and failures.

Latest verification:

  • CI skill regression suite: 11 tests passed.
  • Scout host-side compile check passed for the new CI router and types.
  • Live Scout smoke test passed for /health and /scrape; separate /ci/scrape route was removed after architecture review.

Build 2: Weekly Quality Gate

Goal:

Stop weekly reports from making strategic claims from baseline-only signals.

Tasks:

  1. Detect baseline-only weekly windows.
  2. Prevent battlecard/action recommendations unless there are material deltas.
  3. Add tests for weekly baseline-only behavior.

Done when:

  • A weekly report with zero material deltas says so clearly.
  • No battlecard update is recommended from baseline-only evidence.

Build 3: Private Dashboard v1

Goal:

Turn the static artifact into a usable private dashboard.

Tasks:

  1. Serve dashboard locally/private tunnel only.
  2. Add report detail links.
  3. Add source-health filters.
  4. Add action status updates.
  5. Add archive browsing.

Done when:

  • Arijit can open the dashboard through a private tunnel.
  • Today, Weekly, Sources, Signals, Actions, and Archive are usable without reading raw files.

Build 4: Dedicated CI Bot

Goal:

Move CI delivery away from Athena.

Tasks:

  1. Create/approve dedicated CI Telegram bot.
  2. Configure delivery without exposing it publicly.
  3. Route daily and weekly CI through CI bot.
  4. Send Athena a separate quality/health summary.

Done when:

  • CI bot delivers daily/weekly reports.
  • Athena supervises quality, not daily delivery.

Build 5: Deep Research And Product Upgrade

Goal:

Continue deep market research as an ongoing product-strategy input.

Tasks:

  1. Keep vendor benchmark current.
  2. Add win/loss, review intelligence, buyer-intent, and analyst workflow research.
  3. Translate only differentiated needs into product requirements.

Done when:

  • CI quality rubric is measurable.
  • Dashboard supports decisions, not just browsing.
  • Recommendations are owner-specific and consistently useful.

V1 Definition Of Done

CI Command Center v1 is done when:

  • Daily reports are baseline-aware and source-health-aware.
  • Weekly reports distinguish material deltas from baseline coverage.
  • Source health is visible and actionable.
  • Reports are archived and browsable.
  • Actions are trackable by owner and status.
  • Dashboard is private and usable.
  • Dedicated CI bot delivers reports.
  • Athena receives oversight summaries.
  • Stakeholder access remains disabled until auth and claim-quality controls exist.

Non-Negotiables

  • No material claim without a source URL.
  • No quiet-day confidence without source-health context.
  • No battlecard update from baseline-only signals.
  • No stakeholder export in v1.
  • No public dashboard exposure.
  • Athena supervises; CI bot delivers.
  • MyOS Command Center waits until CI instruments are reliable.

Immediate Next Action

Improve semantic delta extraction and report validity so the prototype starts serving the narrow wedge:

  • What changed?
  • Why does it matter to Algolia?
  • Who should act?
  • What evidence supports it?
  • Is it worth executive attention?