Dashboard/Chowmes/logs/2026-06-29-algolia-inference-low-end-routing.md
Chowmes

logs/2026-06-29-algolia-inference-low-end-routing.md

Algolia inference low-end routing

Date: 2026-06-29 Status: Implemented on Chowmes VPS

Decision

Chowmes now uses a tiered model route:

  • Gemini Flash remains the default Athena/Hermes model for normal Telegram work and content-critical synthesis.
  • Gemini Pro remains the complex/deep escalation model for judge, boardroom, coding, and delegated deep work.
  • Algolia inference is the low-end fast lane for explicit fast/casual use and cheap background housekeeping.

Live routing

  • Default: gemini:gemini-2.5-flash
  • Complex/delegated: gemini:gemini-2.5-pro
  • Low-end explicit modes: /model fast, /model casual
  • Low-end auxiliary tasks: title_generation, triage_specifier, profile_describer, monitor, skills_hub

Credential boundary

The Algolia inference secret stays in /opt/data/.env and profile .env files. The Hermes config references the provider as algolia-inference; it does not store the API key in YAML or markdown.

Required env vars:

  • ALGOLIA_INFERENCE_BASE_URL
  • ALGOLIA_INFERENCE_API_KEY
  • ALGOLIA_INFERENCE_MODEL

Verification

  • Local curl smoke against Algolia inference returned HTTP 200 with OpenAI-compatible choices.
  • Chowmes container curl smoke returned HTTP 200.
  • scripts/chowmes-provider-credit-watch now validates Algolia inference when CHOWMES_CHECK_ALGOLIA_INFERENCE=1.
  • Hermes /model fast smoke returned ALGOLIA_ALIAS_OK.
  • Hermes /model casual smoke returned ALGOLIA_CASUAL_OK.
  • Hermes /model judge smoke returned GEMINI_PRO_OK.
  • Hermes default smoke returned GEMINI_DEFAULT_OK.

Rationale

This avoids using paid Gemini capacity for trivial model work while keeping Athena's normal personality/judgment and CI synthesis on a stronger model. Algolia inference is useful as a fast lane, but it should not be treated as the default CEO brain until its reasoning quality, safety behavior, and personality preservation are proven in live Telegram use.

Guardrails

  • Do not route vision to Algolia inference unless the endpoint is confirmed multimodal.
  • Do not route approval/safety checks to Algolia inference without a safety-specific smoke test.
  • Do not route CI synthesis to Algolia inference until report quality is benchmarked against Gemini.
  • Do not route Athena default Telegram to Algolia inference unless Arijit explicitly accepts the quality tradeoff.