Video-Editing

Wiki/Architecture.md

Architecture — Demo Factory

Last updated: 2026-05-04 (Session 2) Source: Spec v1.1, SCRATCHPAD D0–D17

Plugin structure

Video_Editing/
├── plugin.json                    # Claude Code plugin manifest
├── skills/
│   ├── demo-factory/SKILL.md      # Main orchestrator — activates pipeline, conducts gates
│   ├── synopsis/SKILL.md          # LLM-1 (Sonnet): vault docs → synopsis.md
│   ├── script-writer/SKILL.md     # LLM-2 (Opus): synopsis + screens → script.json
│   ├── capture/SKILL.md           # LLM-3 (Haiku): browser interaction log → capture.yaml
│   └── brand-extractor/SKILL.md   # LLM-4 (Haiku): CSS/favicon/meta → brand.yaml
├── commands/demo/                 # /demo slash command handler
├── scripts/                       # Deterministic TypeScript — zero LLM
│   ├── workspace.ts               # Run state machine + vault workspace management
│   ├── capture.ts                 # Playwright runner (replay + record modes)
│   ├── brand-detect.ts            # CSS vars + favicon + meta scraper
│   ├── render.ts                  # Remotion CLI wrapper; embeds assets as base64
│   ├── upload.ts                  # StorageProvider (Google Drive Phase 1)
│   ├── validate.ts                # Zod + JSON Schema validators for all artifact types
│   ├── readiness.ts               # Read-only audit: server up? capture.yaml? brand? docs?
│   └── preflight.ts               # ffmpeg/Node version hard-fail checks
├── engine/                        # Remotion compositions (React for video)
│   └── src/
│       ├── DemoComposition.tsx    # Root composition; aspect-aware (9:16 / 16:9 / 1:1)
│       ├── layers/
│       │   ├── ScreenLayer.tsx    # Screenshot with browser chrome card; objectFit:cover
│       │   ├── TextOverlayLayer.tsx # Frosted-glass pill; spring animation entry
│       │   ├── VOPlayer.tsx       # Audio playback (11Labs MP3)
│       │   └── AvatarLayer.tsx    # Heygen avatar video overlay
│       └── themes/BrandContext.tsx # Per-product brand via React context
├── adapters/
│   ├── voice/elevenlabs.ts        # VoiceProvider: auto (API) + export (manifest) modes
│   ├── avatar/heygen.ts           # AvatarProvider: export mode Phase 1
│   └── storage/gdrive.ts          # StorageProvider: Google Drive upload
├── cli/demo-factory.ts            # Standalone binary: preflight / readiness / build
├── lib/logger.ts                  # Structured logger (replaces console.log)
└── types/run.ts                   # RunState, CaptureManifest, Script, Beat — all Zod-validated

LLM discipline — 4 synthesis points only

Everything outside these 4 points is deterministic TypeScript with validated schemas.

Point	Model	Input	Output
LLM-1: synopsis	Sonnet	Vault docs + repo README	`synopsis.md`
LLM-2: script-writer	Opus	synopsis.md + capture manifest	`script.json` (multi-track)
LLM-3: capture	Haiku	Browser interaction log	`capture.yaml`
LLM-4: brand-extractor	Haiku	CSS vars + favicon + meta tags	`brand.yaml`

All 4 skills created via skill-creator:skill-creator — no hand-written skill files.

Data flow

User invokes /demo or CLI build
  → preflight.ts: ffmpeg + Node version checks (hard fail)
  → workspace.ts: create run directory in vault + local binary path; write run.json (status: init)
  → capture.ts: replay capture.yaml via Playwright → screenshots as base64 data URIs
  → (Gate 1: user reviews screenshots)
  → synopsis skill (LLM-1): vault docs → synopsis.md → run.json (status: synopsized)
  → (Gate 2: user reviews synopsis)
  → script-writer skill (LLM-2): synopsis + manifest → script.json → run.json (status: scripted)
  → (Gate 3: user reviews script; 30–60s budget enforced)
  → render.ts: Remotion bundle → headless Chrome render → MP4 per format
  → (Gate 4: user reviews video; re-render if needed)
  → upload.ts: MP4 → Google Drive → vault INDEX.md updated
  → run.json (status: done)

Remotion rendering

Remotion uses React components compiled to video. Key constraint: - Assets must be base64 data URIs in inputProps — never use publicDir/HTTP serving. Webpack's bundle cache is unreliable across runs; HTTP 404s on screenshot assets were the root cause of the white-screen bug (fixed Session 2).

Composition parameterization: - script.json + brand profile → inputProps - Each beat maps to a time range; screen_ref points to a scene in the capture manifest - Layers are stacked: ScreenLayer (bottom) → TextOverlayLayer → VOPlayer → AvatarLayer (top) - DemoComposition is aspect-aware; layout primitives swap based on format prop

Storage layout

Vault (text — syncs across machines):
  Projects/Video-Editing/
    Products/<slug>/
      capture.yaml          # Playwright replay script
      brand.yaml            # per-product brand override
    Runs/<run-id>/
      run.json              # pipeline state
      synopsis.md
      script.json

Local disk (binaries — fast IO, no sync):
  ~/AI-Development/Video_Editing/binaries/<slug>/<run-id>/
    captures/               # raw screenshots
    audio/                  # 11Labs MP3 files
    renders/9-16/demo.mp4   # final render

Google Drive (finals — shareable URL):
  AI-Development/Video_Editing/demos/<run-id>/demo.mp4

Provider interfaces

All external services implement a provider interface — swap adapters without engine changes:

Interface	Phase 1 impl	Mode
`VoiceProvider`	`elevenlabs.ts`	`auto` (API) or `export` (manifest only)
`AvatarProvider`	`heygen.ts`	`export` mode Phase 1
`StorageProvider`	`gdrive.ts`	Google Drive CloudStorage mount

Integration points

External	Connection	Purpose
Playwright	npm package	Screenshot capture (replay + record)
Remotion	npm package in `engine/`	React → MP4 render
11Labs API	`ELEVENLABS_API_KEY` env	VO audio generation
Heygen	Export bundle (manual)	Avatar video
Google Drive	CloudStorage mount at vault path	MP4 archive

Known fragile points

CLI build stops after capture — render step not wired (KI-001). Synopsis + script-writer are skill invocations (conversational), not script calls — CLI is deterministic-only.
Sub-skill quality unknown — synopsis + script-writer have SKILL.md files but haven't run against real product content yet.
Heygen in export mode — no API automation in Phase 1; user pastes bundle into Heygen UI manually.