User Flows — heygen-avatar

---

Flow 1: CLI Persona Setup (Developer, first-time)

Who: Developer setting up a new avatar-enabled app for the first time.

1. Run npx heygen-avatar init in project root
2. CLI prompts for: - Agent name - Role/title - Tone (friendly / professional / formal / casual / authoritative) - Character traits / speaking style - Primary conversation goal (answer-questions / guide / train / support / general) - Topics allowed - Topics to refuse/redirect - Response length (short / medium / detailed) - Specific phrases or behaviors to avoid
3. CLI compiles answers into a PersonaConfig object
4. CLI writes heygen.config.ts to the project root
5. Developer imports config from heygen.config.ts and passes to <HeygenAvatar> or useHeygenAvatar

---

Flow 2: Drop-in React Component (Next.js App Router)

Who: Developer adding an avatar to an existing Next.js app.

Backend (one file):

// app/api/heygen/token/route.ts
import { createHeygenSessionHandler } from 'heygen-avatar/server'
export const POST = createHeygenSessionHandler({ apiKey: process.env.HEYGEN_API_KEY! })

Frontend (one file):

// app/page.tsx
'use client'
import { useRef } from 'react'
import { HeygenAvatar } from 'heygen-avatar'
import type { AvatarHandle, HeygenConfig } from 'heygen-avatar'
import myConfig from './heygen.config'

export default function Page() {
  const ref = useRef<AvatarHandle>(null)
  return (
    <HeygenAvatar
      ref={ref}
      sessionTokenUrl="/api/heygen/token"
      config={myConfig}
      style={{ width: 640, height: 480 }}
    />
  )
}

User journey: 1. Page loads → component auto-starts session 2. Status transitions: idle → loading → ready 3. Avatar video renders when ready 4. User speaks (if voiceChat started) or app calls ref.current.speak(...) 5. ref.current.stop() ends the session

---

Flow 3: Headless Hook (Custom UI)

Who: Developer who wants full UI control.

const { videoRef, status, speak, interrupt, startVoiceChat } = useHeygenAvatar({
  sessionTokenUrl: '/api/heygen/token',
  config: myConfig,
  autoStart: true,
  onReady: () => console.log('avatar ready'),
})

return (
  <>
    <video ref={videoRef} autoPlay playsInline />
    <button onClick={() => speak({ text: 'Hello!', taskType: TaskType.REPEAT })}>
      Speak
    </button>
    <button onClick={interrupt}>Interrupt</button>
  </>
)

---

Flow 4: Python Backend (FastAPI)

Who: Developer using a Python backend instead of Next.js API routes.

# main.py
import os
from fastapi import FastAPI
from heygen_avatar import fastapi_route

app = FastAPI()
app.post("/api/heygen/token")(fastapi_route(api_key=os.environ["HEYGEN_API_KEY"]))

Frontend still uses sessionTokenUrl="/api/heygen/token" — the client doesn't care whether the backend is Node or Python.

---

Flow 5: Knowledge Ingestion (TypeScript)

Who: Developer building an avatar with a rich knowledge base.

import { ingest } from 'heygen-avatar'

const knowledge = await ingest()
  .addText('Our product is an AI-powered search platform...')
  .addUrl('https://docs.example.com/overview')
  .addPdf(pdfBuffer)
  .build({ maxWords: 5000 })

// Pass the assembled knowledge string to config
const config: HeygenConfig = {
  avatarName: 'Wayne_20240711',
  mode: 'full',
  persona: { ... },
  knowledge: { sources: [{ type: 'text', content: knowledge }] },
}

---

Flow 6: Lite Mode (BYOLLM)

Who: Developer wanting to use their own LLM (OpenAI, Anthropic, Groq, etc.) instead of HeyGen's built-in GPT-4o-mini.

const config: HeygenConfig = {
  avatarName: 'Wayne_20240711',
  mode: 'lite',
  persona: { ... },
  liteConfig: {
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-4o',
    // baseURL: 'https://api.groq.com/openai/v1'  // or any OpenAI-compatible endpoint
  },
}

What happens internally: 1. attachLiteLoop(sdk, { provider, systemPrompt }) called after SDK init 2. Avatar listens for user speech via USER_TALKING_MESSAGE / USER_END_MESSAGE 3. On USER_END_MESSAGE: accumulated transcript → LLM call → avatar.speak(reply) 4. Conversation history maintained in memory for context continuity 5. liteCleanupRef.current() on unmount removes event listeners