Dashboard/Video-Editing/Specs/SCRATCHPAD.md
Video-Editing

Specs/SCRATCHPAD.md

Product Demo Factory — Requirements Scratchpad

Live notes during brainstorming. Will harden into a spec at Projects/Video-Editing/Specs/YYYY-MM-DD-product-demo-factory-design.md once approved.

Vision (long-term)

This is not just a skill — it's an entire framework / distribution / execution system. Likely shape: - A plugin containing skills + scripts + agents + MCP wiring - Becomes a capability of ArijitOS focused on video editing / product-demo building / content distribution - Composable: each capability is independently usable AND chains into bigger flows

The current scope is Phase 1 — quick, flashy product demo videos for Arijit's own products: Crawler Factory, Algolia Central, PRISM, NoCo II. Everything else is captured under Future Extensions below.

Naming

Working name: Product Demo Factory (Phase 1). Parent system / future name candidates: Origin OS — Video & Demo, ArijitOS Demo Studio, Demo Factory. To be decided.

Phase 1 Scope — Product Demo Factory

A Claude Code skill that generates 30–60 second flashy product demo videos from: - A running local dev server (URL the user provides) - Project documentation (the user provides path or synopsis)

Pipeline (high level): 1. Capture — Skill drives a browser, navigates the running app, takes screenshots 2. Synthesize — Pulls a synopsis from the project's documentation (or accepts a user-provided synopsis) 3. Script — Suggests a 30–60s demo script based on synopsis + screen captures 4. Assemble — Produces an animated product demo: screens + overlays + transitions + animation 5. Voice/avatar feed (optional) — Same skill outputs script + screen assets formatted for: - 11Labs — script → voiceover audio - Heygen — script + screenshots → avatar talking-head with screens 6. Final video — Either standalone (skill renders directly) or composited with Heygen/11Labs output

Three Standalone Modes

The skill must be usable in any of these modes independently:

  1. Full standalone — capture → script → render product demo video end-to-end
  2. Script generator — project docs → demo script (no rendering, just text artifact)
  3. Feeder — produce script + screenshot bundles formatted for Heygen / 11Labs ingestion

Quality Bar

  • 30–60 seconds
  • Flashy, animated
  • Screen recordings / screenshot showcase with motion
  • Optional avatar overlay (Heygen) — lower-third, picture-in-picture, or fullscreen modes
  • Optional voiceover (11Labs)
  • On-brand (Arijit's brand / each product's brand)

Target Products (initial test set)

  • Crawler Factory
  • Algolia Central
  • PRISM
  • NoCo II

Open Requirements (to capture in brainstorming)

  • [ ] Trigger model — slash command? skill activation phrase? CLI?
  • [ ] Capture mechanism — Playwright MCP? Custom? Multi-page navigation strategy?
  • [ ] Documentation source — vault path? README? user-pasted synopsis? all three?
  • [ ] Script style — single voice narration? bullets-on-screen? hooks/CTAs?
  • [ ] Render engine — Remotion (recommended from research), or alternative?
  • [ ] Avatar integration — does Heygen ingest screenshots + script directly, or compose in Heygen UI?
  • [ ] Voice integration — 11Labs API direct? Or export script as text for manual paste?
  • [ ] Output format — MP4 1080×1920 (vertical/Reels)? 1920×1080 (landscape/YouTube)? Both?
  • [ ] Storage / archive — where do generated demos live?
  • [ ] Iteration model — single shot? user reviews and re-prompts? scene-by-scene approval?
  • [ ] Brand assets — logos, colors, fonts per product. Where stored, how injected?
  • [ ] Failure modes — what if dev server isn't running? screenshots are blank? script too long?

UX Flow Questions (to brainstorm)

  • [ ] What does "kicking off a demo" feel like for Arijit? Single command? Wizard? Conversation?
  • [ ] How much control vs autopilot? (Approve script before render? Approve screenshots before script?)
  • [ ] Where does Arijit see progress? CLI text? Live preview?
  • [ ] Where does the final artifact land? Vault? Local folder? Auto-uploaded?

Decisions (locked as we go)

D0 — Execution vehicle: Claude Code plugin (Option A)

  • Locked: 2026-05-03
  • Builds as a Claude Code plugin (~/.claude/plugins/demo-factory/)
  • NOT a standalone SaaS or desktop app (Phase 1)
  • Runs in: Claude Code Terminal ✅, Claude Code Desktop ✅ (same ~/.claude/ install covers both)
  • Open question: Does "Claude Cowork" mean claude.ai web (different surface, needs API bridge) or just distributing the plugin to teammates? Pending user answer.

D1 — Distribution shape: Plugin (all three invocation surfaces)

  • Locked: 2026-05-03
  • The Demo Factory ships as a Claude Code plugin, not a single skill
  • Single underlying engine, exposed three ways:
  • Slash command — /demo <product> <dev-server-url> for fast CLI invocation
  • Conversational skill — auto-activates from natural language ("build a demo for X")
  • Standalone CLI — demo-factory ... runnable outside Claude Code (cron/git hooks/manual)
  • Why: Aligns with the framework / Origin OS reframe. Builds the engine once, three reach paths from day one.
  • Implication: Repo structure must be plugin-shaped (plugins/demo-factory/ with skills/, commands/, plus a bin/ or cli/ for the standalone tool, and a shared engine/ library both consume).

D2 — Pipeline shape: Stage gates with 4 approval points

  • Locked: 2026-05-03
  • The engine is a stateful pipeline with explicit checkpoints. User approves before moving to the next stage: 1. Capture — review captured screens, prune / re-shoot before continuing 2. Synopsis — review/edit doc-derived synopsis 3. Script — review/edit final 30–60s script 4. Render — final video review (re-render if needed)
  • Why: Catches errors cheap. Render is the most expensive stage (time + possibly Heygen/11Labs API spend); approving upstream artifacts first protects that spend.
  • Implications:
  • Per-demo workspace on disk with each stage's output as a first-class artifact (already aligned with Future Extensions architectural implication)
  • Each gate is resumable — user can rerun a stage without restarting from scratch
  • Each gate must be bypassable via a flag (--yolo / --skip-gates) once the user trusts the output
  • Pipeline state machine: init → captured → synopsized → scripted → rendered → done. Each transition writes a state file in the workspace.

D3 — Capture strategy: Hybrid record-then-replay

  • Locked: 2026-05-03
  • Default: if a capture.yaml (or .ts) exists for the product, replay it via Playwright (deterministic, re-cuttable)
  • Fallback (first run): drop into live-record mode — user drives a real browser, skill records every navigation/click + screenshots
  • After live-record: skill auto-generates the capture script from the recording, persists it to the product's workspace for re-use
  • Why: First demo for a new product is fast (just drive the browser yourself, no setup). Every subsequent demo is one-click replay. Auth and dynamic state work because the user drives on first run. The auto-generated capture script doubles as documentation of how the product demos.
  • Implications:
  • Capture engine has two modes: replay and record. Same screenshot/annotation pipeline downstream.
  • Each product gets a workspace folder (likely Projects/Video-Editing/Products/<product-slug>/) holding capture.yaml, brand assets, past demo runs.
  • Recording must capture: navigation events, click selectors, form inputs, wait conditions, screenshot timestamps. Replay must reproduce timing.
  • Need a way to edit the auto-generated script (it won't be perfect on first record).

D4 — Documentation source: Layered (Vault → Repo → Pasted)

  • Locked: 2026-05-03
  • Skill resolves product context in priority order: 1. Vault — Projects/<product>/ (PRD, Specs, README, etc.) 2. Repo docs — README, CLAUDE.md, package.json, recent commits, if a repo path is provided 3. User-pasted synopsis — fallback if neither exists or both are insufficient
  • At the Synopsis gate (Stage 2 of the pipeline), skill shows the user exactly what it pulled and from where, so the user can edit, augment, or paste a fresh synopsis.
  • Why: All four target products live in the vault, so the default path is rich and zero-effort. Layering keeps the skill portable to non-vault setups (future distribution to other users) and resilient when vault entries are stale.
  • Implications:
  • Need a product registry / locator — given a product slug ("crawler-factory"), find its vault path, optionally its repo path, and any brand assets.
  • Synopsis extraction is itself a small subsystem — likely a sub-skill that knows how to summarize multi-doc inputs into a 2–3 paragraph synopsis appropriate for a 30–60s demo.
  • The Synopsis gate UX must surface provenance: "Pulled from vault Projects/Crawler-Factory/PRD/v2.md (last modified 2 days ago) + repo README.md."

D5 — Script shape: Multi-track (VO + Text-overlay + Avatar)

  • Locked: 2026-05-03
  • The script is a structured object with three parallel tracks, each beat addressing one or more tracks:
  • VO_TRACK — voiceover lines (feed → 11Labs)
  • TEXT_OVERLAY_TRACK — kinetic typography lines (rendered into video by Remotion)
  • AVATAR_TRACK — avatar talking lines (feed → Heygen)
  • The script also encodes the structural beats for the demo: Hook → Problem/Setup → Demo (per scene with screen reference) → CTA.
  • Per-render output mode: user picks which tracks are active for each render variant. Same source script can produce:
  • Silent kinetic-text reel (TEXT_OVERLAY only) — Twitter, LinkedIn autoplay
  • Narrated explainer (VO + TEXT_OVERLAY) — YouTube, web embeds
  • Avatar-led (AVATAR + TEXT_OVERLAY) — LinkedIn founder-voice posts
  • Full stack (all three) — premium / hero distribution
  • Why: Phase 1 must feed 11Labs, Heygen, AND render standalone. A multi-track script is the artifact shape that powers all three without re-writing per channel. Also unlocks F6 (Multi-Channel Distribution Pack) cleanly — same script object, different track activations per channel.
  • Implications:
  • Script schema is a first-class type (likely TypeScript / JSON Schema). Validated at the Script gate.
  • Beat structure: each beat has id, start_ms, duration_ms, screen_ref (which captured screen), and any of vo, text_overlay, avatar_line.
  • The 30–60s budget is enforced at the script gate — engine flags if total duration_ms exceeds 60000.
  • Script generation is itself a sub-skill that takes synopsis + capture manifest and produces the multi-track script.
  • Renderer is track-aware — composition reads the active tracks and assembles only those layers.

D6 — Render engine: Remotion (Phase 1) with hosted-b-roll deferred to Phase 2

  • Locked: 2026-05-03
  • Phase 1 renderer: Remotion (React for video). Official Claude skill installed as a sub-dependency (npx skills add remotion-dev/skills).
  • Why Remotion: Strongest Claude integration in the 2026 ecosystem (28 modular rules, 6M-view launch, 25K installs week one). Programmatic, deterministic, re-rendererable from the same source — exact match for the "show this exact screen at this second" requirement. Lets us reuse React components across all four products' brand profiles.
  • Why not hosted video models for the core demo: Non-deterministic, can't reliably composite real product screenshots, hard to re-cut when the product evolves.
  • Phase 2 extension (deferred): Hosted video models (Veo / Sora / Kling via Higgsfield MCP) generate b-roll only — transitions, atmospheric shots, intro/outro stings — composited into the Remotion timeline. Lives under Future Extension F4.
  • Implications:
  • Engine ships a <DemoComposition /> family of Remotion compositions parameterized by the script object + screen manifest.
  • Each track from D5 maps to a Remotion layer (<VOPlayer />, <TextOverlayLayer />, <AvatarLayer />, <ScreenLayer />).
  • Renders are deterministic given the same script + assets, so re-cuts are free.
  • Need a package.json and Remotion bootstrap inside the plugin's engine/ workspace.

D7 — Output formats: C architecturally + D as default policy

  • Locked: 2026-05-03
  • Architecturally: Engine and Remotion compositions support all three aspect ratios as first-class:
  • 9:16 vertical — 1080×1920 — Reels / Shorts / TikTok / LinkedIn vertical
  • 16:9 landscape — 1920×1080 — YouTube / web embeds / sales decks
  • 1:1 square — 1080×1080 — in-feed Twitter / LinkedIn
  • Default render policy: Only 9:16 is produced unless the user explicitly opts in to more. CLI / skill flag:
  • --format=9:16 (default)
  • --format=16:9
  • --format=1:1
  • --format=all (full multi-channel pack — aligns with F6)
  • Why: Vertical-first is where most demos land in 2026. But adaptive composition logic must exist from day one — retrofitting aspect ratios into pixel-fixed layouts is painful. Building it adaptive once, defaulting to one, lets us turn on F6 (Multi-Channel Distribution Pack) with zero architectural churn.
  • Implications:
  • Compositions are aspect-aware — same component tree, different layout primitives (<VerticalLayout>, <LandscapeLayout>, <SquareLayout>) wrapping shared scene content.
  • Screen captures must be high enough resolution to crop / scale into any aspect (capture at 2x or 3x and downscale).
  • Text overlay sizing, screen-window scaling, avatar position all parameterize on aspect.
  • Per-format render times tracked separately so user can budget.

D8 — External integrations: Pluggable providers

  • Locked: 2026-05-03
  • Engine defines two provider interfaces:
  • VoiceProvider — generates VO audio from a VO track (input: script.vo_track + voice_profile, output: audio file path + timing manifest)
  • AvatarProvider — generates avatar video from an avatar track (input: script.avatar_track + avatar_profile + screen refs, output: video file path)
  • Phase 1 adapters shipped:
  • 11Labs (VoiceProvider) — supports auto (API mode) and export (manifest only) modes
  • Heygen (AvatarProvider) — supports auto (API mode) and export (bundle for manual UI paste) modes
  • Default config (Phase 1): 11Labs in auto mode (cheap, stable, fast); Heygen in export mode (expensive, slower, controlled). User can flip per-render via flag or config.
  • Why: "Feeder" requirement satisfied either way. Pluggable interfaces let new providers (Synthesia, OpenAI Voice, future avatar tools) slot in as adapters without engine changes. Default config is the pragmatic balance.
  • Implications:
  • API key management — keys stored in env (ELEVENLABS_API_KEY, HEYGEN_API_KEY) or in a local ~/.demo-factory/credentials file. Never in vault, never in repo.
  • Rate-limit and failure handling lives in each adapter, not the engine.
  • Billing-aware UX: before any auto-mode call, skill estimates cost from script length and surfaces it to user (e.g., "11Labs estimate: ~120 chars × $0.0003 = ~$0.04. Proceed?"). Cost-warning gate especially important for Heygen.
  • VoiceProfile and AvatarProfile are first-class config types (per-product or per-persona).
  • Export bundle format must be standardized so any adapter can produce it: a folder with manifest.json describing what the user should paste/upload where, plus the assets.

D9 — Brand profile: Autodetect → user-edit → persistent override (with shared base)

  • Locked: 2026-05-03
  • Two-layer brand model:
  • Base profile — ~/.demo-factory/brand/arijitos-base.yaml — house style: default font (display + body), default avatar_id (Heygen), default voice_id (11Labs), accent color, tone defaults. Applies if not overridden.
  • Per-product override — Products/<slug>/brand.yaml — only fields that differ from base (typically primary/accent colors, tagline, product-specific voice if any).
  • First-run flow: 1. Skill scrapes the running app's CSS, favicon, meta tags 2. Builds an inferred brand profile (colors, font hints, logo path) 3. Shows it to user with a diff against the base 4. User confirms / edits / approves 5. Result persisted as Products/<slug>/brand.yaml
  • Subsequent runs: Replay from the saved override merged onto base. Skill flags if the running app's CSS has drifted from the saved profile (offers re-detect).
  • Why: Zero-config first demo. Consistent Arijit aesthetic (base) across all products. Each product still gets its own visual identity (override). Aligns with vision of a unified "Arijit-as-builder" voice across Crawler Factory, Algolia Central, PRISM, NoCo II.
  • Implications:
  • Brand profile schema is a first-class type: colors, fonts, logo refs, voice_id, avatar_id, tone, tagline, dark/light variants.
  • Brand asset extractor is a sub-skill (CSS parser, favicon → logo conversion, meta-tag reader).
  • Remotion compositions consume the merged profile via React context — every layer inherits brand without hardcoding.
  • Brand drift detection on subsequent runs (compare saved profile against newly-scraped CSS) — surface as a warning, not a block.

D10 — Storage: Split vault/local + cloud archive for finals

  • Locked: 2026-05-03
  • Three storage tiers:
  • Vault (text) — Projects/Video-Editing/Products/<slug>/ and Projects/Video-Editing/Runs/<run-id>/ hold all text artifacts: brand.yaml, capture.yaml, synopsis.md, script.json (multi-track), run-log.md, render-manifest.json. Searchable in Obsidian, syncs across machines.
  • Local disk (binaries) — ~/AI-Development/Video_Editing/binaries/<slug>/<run-id>/ holds heavy assets: raw screenshots, captured DOM snapshots, 11Labs audio MP3, Heygen avatar MP4, intermediate Remotion frames, draft renders. Fast IO, no sync overhead.
  • Cloud archive (finals) — final approved MP4s auto-uploaded to a cloud bucket. Vault YAML stores both local path + cloud URL. Distribution-ready, accessible across machines.
  • Cloud target: TBD. Decision needed: Google Drive (existing CloudStorage mount) vs S3/R2 vs vendor-neutral. Capturing as open sub-decision SD-1 below.
  • Why: Vault stays searchable and sync-healthy (text only). Local binaries are fast. Final MP4s have a permanent shareable URL — supports F8 (Demo Library) and F6 (Multi-Channel Distribution Pack). User explicitly chose cloud archive in Phase 1, not deferred.
  • Implications:
  • Run-id scheme — every demo invocation gets a stable id (e.g., 2026-05-03-crawler-factory-001). Vault and local paths both keyed by it.
  • Index file in vault — Projects/Video-Editing/Runs/INDEX.md auto-maintained list of all runs with product, status, render formats, cloud URLs.
  • Cloud uploader is a small subsystem with the same pluggable shape as voice/avatar adapters (StorageProvider interface) so users can pick their cloud.
  • Cleanup strategy — local binaries can be pruned safely once the final MP4 is in the cloud archive. Runs older than N days → prune binaries, keep text + cloud URL.

Open Sub-Decisions (to revisit)

SD-1 — Cloud archive target

Pick the cloud destination for final MP4s. Candidates: - Google Drive (existing CloudStorage mount, no new account, but quota is shared with vault) - AWS S3 / Cloudflare R2 (vendor-neutral object storage, cheap, public URLs trivial) - YouTube Unlisted (free hosting + native distribution, but a render target, not just storage) - Vimeo / Loom (purpose-built, watermark concerns)

Defer to design phase or short follow-up question.

Risks / Tigers

  • Capture quality: dynamic apps (auth, state) may need scripted flows
  • Heygen / 11Labs API access — paid tiers, API keys, rate limits
  • Brand consistency across 4 different products
  • Render time and cost per demo

Future Extensions (parking lot — do not lose the thought)

These are out of Phase 1 scope but are part of the long-term system. Capture them here so Phase 1 architecture doesn't paint us into a corner.

F1. UGC Ad Creation

Generate UGC-style ads from the same source material (product + docs + screenshots). Different tone, different pacing, different actor framing than product demos.

F2. Nano Banana Image Generation (Presentation Assets)

From the script and the demo's narrative beats, generate presentation-grade images via Nano Banana (Gemini 3 image generation). Use cases: - Hero images for the demo's distribution channels - Slide visuals when the demo is converted into a deck - Thumbnail / cover art

F3. Visual Storyboard / Story Pictures

Represent the script's narrative as a sequence of pictures (storyboard) usable in a presentation or carousel. Each script beat → one image. Used both as a creative asset for slides and as input to downstream video generators (Sora / Veo / Kling) if we move into AI-generated b-roll.

F4. AI Video Generation (Nano Banana → Veo / Sora / Kling)

From script + storyboard images, generate animated video clips to be used as b-roll, transitions, or scene overlays. Layered on top of the screen-recording demo to amplify production value.

The same script + screens + storyboard images repackaged as a LinkedIn carousel post (multi-slide PNG/PDF). The demo video is the hero asset; the carousel is the distribution asset.

F6. Multi-Channel Distribution Pack

For each demo, automatically produce a distribution bundle: - Vertical 9:16 demo video (Reels / Shorts / TikTok) - Horizontal 16:9 demo video (YouTube / Twitter / LinkedIn video) - Carousel (LinkedIn / Instagram) - Hero image (presentation / thumbnail) - Caption / post copy (per channel) - Hashtag suggestions

F7. Full Video Editing Suite

The original "bigger vision" — clip cutting, editing, multi-source compositing, sound design, color grading. The Demo Factory is one capability inside this larger studio.

F8. Demo Library / Archive

A searchable archive of past demos with metadata: which product, which version, which feature, when shipped. Re-cuts and updates pull from here.

F9. Avatar Library

Manage multiple avatars (Arijit's, brand-specific avatars per product, custom personas) and let the user pick per demo.

F10. Voice Library

Manage 11Labs voice profiles per product or per demo persona. A/B test voices.

F11. "Re-cut" / Update Pipeline

When a product evolves, re-run the demo factory against the updated app and updated docs. Diff old vs new demo so the user can decide what to refresh.

F12. Analytics Loop

After distribution, feed back engagement metrics (views, watch-through, comments) into the Demo Factory so future demos get optimized for what actually performs.

Architectural Implications of Future Extensions

These extensions imply Phase 1 must: - Treat script, screenshots, storyboard, rendered video, and distribution metadata as first-class, separable artifacts (not buried inside a single render) - Use a plugin / framework structure from day one (skills + scripts + commands + MCP wiring), not a single monolithic skill - Stand up a per-demo workspace on disk (or in vault) so every artifact is addressable, re-usable, and re-cuttable later - Keep brand profile, avatar profile, voice profile as injectable config — not hardcoded