Dashboard/Competitive Intelligence/logs/2026-06-27-ci-evidence-basis-and-benchmark.md
Competitive Intelligence

logs/2026-06-27-ci-evidence-basis-and-benchmark.md

CI Evidence Basis And Market Benchmark

Date: 2026-06-27

Why This Exists

Arijit challenged the trust basis of the CI Command Center plan:

  • What is the plan based on?
  • What does "baseline" mean?
  • What does "material delta" mean?
  • Where did the sources come from?
  • What professional CI tools were studied?
  • How do we know the report format is valid?
  • How do we know we are building the right thing?

This note is the explicit answer. It separates evidence, assumptions, product choices, and remaining research gaps.

Honest Status

The prior CI project map was directionally useful but under-evidenced. It was based on:

  1. Arijit's stated intent for a standalone CI product.
  2. The existing Chowmes CI code and source registry.
  3. A live runtime/source audit.
  4. A shallow initial benchmark of CI tools.

That is not enough to claim "best in industry." It is enough to start stabilization, but not enough to define the final product bar. This note upgrades the evidence base and defines the research still required.

What "Baseline" Means

Baseline is the first known captured state of a source.

Examples:

  • First time fetching Constructor's customer page.
  • First time fetching Bloomreach's changelog.
  • First time fetching OpenAI's RSS feed.

Baseline does not mean "not important." It means "not yet proven to be new movement."

Baseline should be used to establish future comparison, not to trigger action.

What "Material Delta" Means

A material delta is a new or changed signal that could affect product, marketing, sales, partnership, or executive decisions.

Examples:

  • A competitor launches or materially changes a feature.
  • A competitor changes pricing or packaging.
  • A competitor publishes new customer proof in a relevant segment.
  • A competitor changes positioning in a way that affects Algolia messaging.
  • An AI-native threat launches a capability that changes the search/discovery category.
  • A source change is semantically understood, not merely detected as "page changed."

Non-material examples:

  • First capture of an existing page.
  • Minor website copy noise.
  • Date/footer/navigation changes.
  • A generic blog post with no competitive implication.
  • A source changed but the system cannot explain what changed.

Where Current Sources Came From

The current source registry came from the existing Competitive Intelligence workspace and Algolia CI pilot. It is a public-source v1 registry, not a complete enterprise CI source strategy.

Current categories:

  • Direct competitor websites: Coveo, Bloomreach, Constructor, Google Vertex AI Search.
  • Traditional/adjacent competitors: Elastic, Lucidworks, Yext, Algonomy.
  • OSS/search threats: Meilisearch, Typesense, AWS OpenSearch.
  • AI-native threats: OpenAI, Perplexity, protocol ecosystem sources.
  • Industry/news sources: CMSWire, Martech Edge, Contra Collective.
  • Community sources: Reddit and Hacker News.
  • Algolia public baseline sources to prevent false gap claims.

Validity:

  • Valid as a public-source pilot.
  • Not valid as a complete CI program.
  • Missing paid/structured sources such as review platforms, analyst reports, traffic data, SEO data, sales calls, CRM, win/loss interviews, and internal field intelligence.

Professional CI Tool Benchmark

Tool Evidence-backed capabilities Implication for Chowmes CI
Klue Combines competitive intelligence and win-loss programs; supports deal-based competitive insights with personalized recommendations; integrates competitive intel into workplace tools. Source: https://klue.com/ Chowmes CI must connect intelligence to decisions and action routing, not just monitoring. Later versions need win/loss and field feedback.
Crayon Monitors competitors, alerts relevant intel, offers inbox priority insights, AI summarization, AI importance scoring, battlecards, announcements, newsletters, win/loss analysis, engagement data, influenced revenue. Source: https://www.crayon.co/ Chowmes CI needs monitoring, scoring, battlecard/update workflow, usage/adoption metrics, and revenue/action impact over time.
Kompyte Tracks competitor updates across websites, reviews, content, social, ads, and job postings; AI filters noise; AI daily summaries; battlecards integrated with CRM/sales tools. Source: https://www.kompyte.com/ Chowmes source strategy must expand beyond websites into reviews, content, social, ads, jobs, and CRM/sales enablement surfaces.
Contify AI-native market and competitive intelligence platform; collects/analyzes/delivers insights from global and proprietary internal sources; offers dashboards, newsletters, newsfeeds, smart filters, AI facts, automated insights, ad-hoc Q&A grounded in curated datasets. Source: https://www.contify.com/platform/ Chowmes CI must support role-specific insights, smart filtering, dashboards, newsletters/bot delivery, and grounded Q&A over curated sources.
AlphaSense Market intelligence platform combining 10,000+ private, public, premium, and proprietary external data sources, including company docs, earnings transcripts, filings, analyst research, press releases, internal research, emails, newsletters, web pages, and RSS. Source: https://www.alpha-sense.com/solutions/market-intelligence-platform/ Chowmes v1 is missing premium analyst/filings/research depth. Long-term CI should support internal and premium corpora if access is approved.
Similarweb Digital intelligence for traffic, visibility, SEO/AEO, marketing, acquisition, GTM, eCommerce, market trends, competitor tactics, consumer behavior. Source: https://www.similarweb.com/ Chowmes should eventually add traffic/share-of-voice/digital performance signals, but not in public-source v1 unless APIs or exports are approved.
Semrush Competitor website analysis, traffic analytics, daily/weekly traffic data, top traffic sources, organic rankings, online presence analysis. Source: https://www.semrush.com/features/competitor-website-analysis-tools/ Chowmes needs SEO/content/traffic evidence for Algolia positioning work, but paid API/export access is a later decision.

Feature Model Derived From Benchmark

The benchmark suggests a mature CI system has these layers:

  1. Source monitoring - Websites, docs, changelogs, pricing, blogs, reviews, social, ads, jobs, analyst/news, community, search/SEO, traffic, CRM, calls, internal notes.

  2. Noise filtering - Baseline detection, duplicate suppression, page-noise suppression, source reliability scoring, confidence penalties.

  3. Delta understanding - What changed, why it matters, who it affects, how confident we are.

  4. Analyst workflow - Review queue, evidence checks, source health, confidence, novelty, materiality, approval status.

  5. Enablement artifacts - Daily pulse, weekly synthesis, battlecards, newsletters, announcements, executive briefs.

  6. Action routing - Product, PMM, Sales Enablement, Partner, Exec, CI operator ownership.

  7. Delivery - Dashboard, bot, Slack/Teams/email/CRM later, private tunnel first.

  8. Feedback and ROI - Usage, accepted/rejected actions, battlecard engagement, influenced deals/revenue later.

What Current Chowmes CI Covers

Capability Current coverage
Public website monitoring Partial
Changelog/docs monitoring Partial
News/blog monitoring Partial
Source health Basic, improving
Baseline detection Implemented for daily material filtering
Delta explanation Weak
Weekly synthesis Operational, quality gate incomplete
Report archive Basic model exists
Dashboard Static prototype exists
Action queue Basic model exists
Dedicated bot Not built
Battlecards Not built
Win/loss Not included
CRM/Gong/Salesforce Not included
Reviews/social/ads/jobs Mostly not included
SEO/traffic intelligence Not included except public content movement
Usage/ROI analytics Not built

Report Format Validity

The current report format is valid only as an internal v1 operator brief.

It is valid because it includes:

  • Bottom line.
  • Recommended action.
  • Evidence.
  • Watch trigger.
  • Artifact links.
  • Source-health caveats.

It is not yet valid as stakeholder-grade CI because it lacks:

  • Full evidence-review workflow.
  • Claim-level confidence.
  • Semantic delta explanation.
  • Battlecard update discipline.
  • Human/AI analyst approval.
  • Export/access controls.
  • Feedback/usefulness tracking.

How We Know If We Are Doing The Right Thing

We do not know by vibes. We know by gates.

Gate 1: Source Truth

  • Every source has a status: healthy, fallback-backed, manual-only, disabled, or failing.
  • Quiet-day confidence depends on source health.

Gate 2: Claim Truth

  • Every material claim has source URL, detection date, source type, confidence, and evidence snippet.
  • Claims without evidence are excluded or marked unsupported.

Gate 3: Delta Truth

  • Reports distinguish baseline, changed page, semantic change, material delta, and strategic pattern.
  • "Page changed" alone is not enough for a strategic recommendation.

Gate 4: Action Truth

  • Every recommendation has an owner and next step.
  • Actions can be accepted, rejected, assigned, completed, or sent back for evidence.

Gate 5: Usefulness Truth

  • Arijit or stakeholders can mark useful/not useful.
  • Rejected insights feed back into scoring.

Gate 6: Benchmark Truth

  • Features are mapped against professional CI systems.
  • Gaps are explicit, not hidden.

Immediate Product Correction

The next work should not be "build more UI."

The next work should be:

  1. Improve semantic delta extraction.
  2. Add weekly baseline-only quality gate.
  3. Add source-type expansion plan based on benchmark: reviews, social, ads, jobs, SEO/traffic, analyst/news, internal win/loss later.
  4. Add report validity rubric into CI tests.
  5. Only then polish dashboard and bot behavior.

Research Gaps Still Open

  • Detailed Klue workflow and battlecard lifecycle.
  • Detailed Crayon data model and AI scoring workflow.
  • Detailed Kompyte source/filter/report workflow.
  • Detailed Contify dashboards/newsletters/query workflow.
  • AlphaSense source taxonomy and analyst workflow.
  • Similarweb/Semrush signal types for Algolia relevance.
  • Pricing/packaging where publicly available.
  • What CI teams actually measure: adoption, influenced revenue, win-rate lift, time saved, decision speed.

Bottom Line

The CI Command Center plan should be treated as a hypothesis until it passes this benchmark and quality rubric.

The stabilization work was still necessary. But the product strategy now needs to be driven by a deeper CI benchmark, semantic delta quality, and report validity gates, not by generic "dashboard plus bot" thinking.