Architecture — heygen-avatar

---

Package Structure

heygen-avatar/              ← npm package
  src/
    config/
      types.ts              — HeygenConfig, PersonaConfig, LiteConfig, KnowledgeSource types
      validate.ts           — validateConfig() — 6 runtime guards before session start

    ingest/
      text.ts               — plain text + markdown extraction
      pdf.ts                — pdf-parse v2.4.5 class-based extraction
      url.ts                — cheerio HTML → plain text
      crawler.ts            — BFS web crawler with page budget
      chunk.ts              — assembleKnowledge() — merge + truncate to maxWords
      index.ts              — IngestPipeline builder (add_text/pdf/url/crawl/build)

    lite/
      prompt.ts             — buildSystemPrompt() — 3-section HeyGen persona format
      openai.ts             — createOpenAIProvider() — OpenAI-compatible LLM client
      loop.ts               — attachLiteLoop() — USER_TALKING_MESSAGE accumulation → LLM → speak

    client/
      types.ts              — AvatarRuntimeConfig, AvatarStatus, AvatarHandle, SpeakOptions
      useHeygenAvatar.ts    — headless React hook — full session lifecycle + Lite mode wiring
      HeygenAvatar.tsx      — <HeygenAvatar> — forwardRef, loading/error states, video

    server/
      types.ts              — ServerConfig
      handler.ts            — createHeygenSessionHandler (Web Fetch API) + withExpress adapter

    cli/
      prompts.ts            — @inquirer/prompts interactive persona wizard (10 questions)
      writer.ts             — writes heygen.config.ts to disk
      index.ts              — Commander program ("heygen-avatar init")

    index.ts                — public npm entry: all client exports + types + SDK enums
    server.ts               — server npm entry (heygen-avatar/server)

heygen_avatar/              ← Python package
  __init__.py               — exports create_session_token, fastapi_route, IngestPipeline
  session.py                — async httpx token fetch + FastAPI route factory
  ingest.py                 — IngestPipeline (text/pdf/url/crawl), word-count aware

dist/                       ← compiled output (tsup)
  cli.js                    — 151 lines, shebang, real Commander program
  index.js / .cjs / .d.ts   — ESM + CJS + types for client entry
  server.js / .cjs / .d.ts  — ESM + CJS + types for server entry

---

Two Modes

	Full mode	Lite mode
Who controls LLM	HeyGen (GPT-4o-mini built-in)	Developer (any OpenAI-compatible API)
Conversation loop	Handled by HeyGen platform	attachLiteLoop() in src/lite/loop.ts
Knowledge delivery	knowledgeBase string → HeyGen injects into system prompt	Same string → injected into BYOLLM system prompt
Cost	1 credit = 30s streaming	1 credit = 1 min streaming (cheaper)
Config	mode: 'full'	mode: 'lite' + liteConfig: { apiKey, model }

---

Session Lifecycle (Full mode)

Mount → autoStart=true → startSession()
  Phase 1 (sync):  new StreamingAvatar({ token: '' })
                   attachHandlers(earlySdk)          ← sync, for test timing
  Phase 2 (async): fetch(sessionTokenUrl, POST)      ← token proxy
                   new StreamingAvatar({ token })
                   attachHandlers(sdk)               ← real SDK
  [Lite only]:     attachLiteLoop(sdk, { provider, systemPrompt })
  Phase 3:         sdk.createStartAvatar({ avatarName, quality, language, knowledgeBase })
  → STREAM_READY fires → video.srcObject = stream → status: 'ready'

---

Token Proxy Pattern

The HEYGEN_API_KEY never leaves the server. The browser calls your backend route (/api/heygen/token), the backend calls HeyGen's /v1/streaming/create_token, and returns only the short-lived session token to the browser.

Browser                 Your Backend              HeyGen API
  POST /api/heygen/token
       ─────────────────►
                          POST /v1/streaming/create_token (HEYGEN_API_KEY)
                               ─────────────────────────────►
                                                              { token }
                               ◄─────────────────────────────
                         { token }
       ◄─────────────────
  StreamingAvatar({ token })

---

Lite Mode Conversation Loop

USER_TALKING_MESSAGE (incremental)  →  pendingTranscript = last message
USER_END_MESSAGE (no message field)  →  drain pendingTranscript
                                        history.push({ role: 'user', content })
                                        provider.complete([system, ...history])
                                        → reply
                                        history.push({ role: 'assistant', content: reply })
                                        avatar.speak({ text: reply, taskType: REPEAT })

Key invariant: USER_END_MESSAGE fires UserTalkingEndEvent which has NO message field (only type + task_id). Content must be accumulated from USER_TALKING_MESSAGE events.

---

Build Pipeline

tsup (dual config array):
  Config 1: { index, server } → ESM + CJS + .d.ts, no banner, sideEffects tree-shaking safe
  Config 2: { cli }           → ESM only, #!/usr/bin/env node banner, direct entry at src/cli/index.ts

Critical: CLI entry points directly at src/cli/index.ts (not via a stub file). "sideEffects": false in package.json would tree-shake any bare re-export stub. See ADR-002.

---

Test Architecture

File	What it tests
tests/config/validate.test.ts	8 guard conditions in validateConfig
tests/ingest/chunk.test.ts	assembleKnowledge merge + truncation + word count
tests/ingest/url.test.ts	HTML → text extraction via cheerio
tests/ingest/pdf.test.ts	PDF text extraction via pdf-parse
tests/lite/loop.test.ts	accumulation pattern + LLM call + speak
tests/lite/openai.test.ts	provider.complete happy path + empty response guard
tests/server/handler.test.ts	token proxy, CORS, 401 on HeyGen failure
tests/client/useHeygenAvatar.test.ts	hook lifecycle, two-phase init, Lite mode wiring
tests/client/HeygenAvatar.test.tsx	component renders, loading state, error state, forwardRef

Totals: 9 test files, 40 TypeScript tests + 7 Python tests = 47 tests, all passing.

---

Package Exports

"exports": {
  ".": {
    "types": "./dist/index.d.ts",
    "import": "./dist/index.js",
    "require": "./dist/index.cjs"
  },
  "./server": {
    "types": "./dist/server.d.ts",
    "import": "./dist/server.js",
    "require": "./dist/server.cjs"
  }
}

types condition must appear before import/require — bundler resolution order. See ADR-005.