VideoClaw — Capabilities Manifest (for agents & LLMs)
This file exists so an autonomous agent can answer, in one read: "Can videoclaw do task X, and with which command?" — without crawling the source.
- CLI binary:
vclaw(npm packagevideoclaw; alias binvideoclaw). - Authoritative, machine-readable command surface:
vclaw schema --json(registered commands and their flags). This doc is the human/agent map; that command is ground truth for flags. - Read-only structured access for agents:
vclaw mcp serve(stdio MCP server — see MCP). - How to work on the codebase (not use it): see
CLAUDE.md+docs/.
1. What videoclaw is
A TypeScript/Node >=20.10 multi-provider AI-video CLI. It turns an idea (or a reference ad, or a script, or a set of stills) into a finished, narrated, music-scored video — every pipeline stage explicit, every artifact machine- readable JSON, and the on-disk projects/<slug>/ tree as the single source of truth. The core registry exposes five video routes (Veo, Seedance, Runway, Dreamina and Magnific) plus separate finishing/upscale backends, with human-in-the-loop review portals.
One-sentence router: if a task is "make / edit / assemble / upscale / narrate / score / review a video or its assets," videoclaw can very likely do it — the question is which route and which command, both answered below.
2. Capability matrix — what it can produce
| Capability | Commands (entry points) | Notes |
|---|---|---|
| Text→video | direct produce / execute; queued pool → cinema-work; render-scenes, auto | Choose the route and execution contract before rendering. |
| Image→video (keyframe i2v) | same, driven by scene keyframes | Start-frame; end-frame interpolation on Seedance/Dreamina/Runway. |
| Reference→video (identity lock) | flow-r2v (Flow), Seedance Asset Library, Dreamina Omni refs | Uses character/reference conditioning; review identity in each output. |
| Whole-storyboard continuity chain | produce --auto-chain → cinema-work | Compiles dependencies without rendering; successor tasks wait for selected predecessor output. |
| Cartoon / talking-character shows | show-bible, show-preflight, voice-clone | Reusable cast+locations+voices world. |
| Narration (TTS) | narrate | gemini-tts / elevenlabs-tts / mureka-tts / nari-tts backends. |
| Multi-speaker dialogue | dialogue | Per-turn voice binding. |
| Sound effects | sfx | ElevenLabs SFX. |
| Music / soundtrack | soundtrack | lyria / lyria3 / flowmusic / suno / mureka; account and backend dependent. |
| Voice reference preparation | voice-clone | Local black-frame+audio reference; generated voice consistency still needs review. |
| Diegetic images / UI stills | gen-image | Go Bananas / OpenAI image models. |
| Motion-graphics overlays | overlay, motion-overlay | Lower-thirds, alerts, speech-synced reels. |
| Style-locked motion graphics | mograph-sheet/pack/render/logos | Motion-sheet style lock + action-only prompt packs -> batch queue; shared visual reference; review generated consistency. |
| Reviewed footage repairs | Rap skill repair_workbench.py, repair_apply.py | Preview approved repairs and apply them through normal selections; see the repair planner. These are skill helpers, not top-level CLI commands. |
| Music videos (beat-synced) | music-video | Vocal-synced, beat-exact assembler. |
| Title cards | title-card | PIL/RAQM lower-third + end-card. |
| Stitch / assemble final cut | assemble (--from-clips) | FFmpeg concat + audio mix. |
| HD/4K upscale (finish) | finish | Topaz (hosted/local), Magnific, Runway Topaz 4K (account/plan dependent). |
| Still-image upscale | image-ops | Magnific precision-v2. |
| Lip-sync | lipsync | OmniHuman. |
| Vertical / square / loop / thumbnail / subtitles | make-vertical, make-square, make-loop, thumbnail, burn-subtitles | Local FFmpeg post, no spend. |
| Character consistency audit | consistency-audit, character-consistency | Vision audit for wardrobe/face drift. |
| Motion-artifact QC | motion-qc | Dense-frame vision QC (vapour, morph, vanish). |
| Durable batch preparation | batch-submit --project <slug> → cinema-work | Submission compiler only; historical batch-monitor/batch-status do not execute new tasks. |
| Structured creative planning | storyboard --film-plan, filmmaking-prompts | Format, purpose, sequences, shot action and contextual performance; shared runtime validates planned packets and local reference freshness. |
| Version-bound final review | review --film-edit / --film-review, publish | Actual media hashes, ordered clips, trims and soundtrack bind full-playback evidence to the delivery. |
| Human-in-the-loop review | review-ui, portal, publish-preview | Editor + client review surfaces. |
| Reference-ad cloning | clone-init, clone-execute (clone-ad is a deprecated spelling) | Analyze a reference ad → reproduce. |
Output post-production without any provider/spend: qc, verify-final, make-vertical|square|loop, thumbnail, burn-subtitles are pure local FFmpeg.
3. Provider routes (video generation)
Live-execution commands route to one of these. vclaw video providers prints local readiness; vclaw video verify-env checks configuration. Neither is proof of current account entitlement, output quality or a successful live generation.
| Route id | Provider | Best for | Auth |
|---|---|---|---|
veo-useapi | Google Veo, through your Google Flow account (plus Omni Flash) | Photoreal, registered-character identity lock, i2v | USEAPI_API_TOKEN + USEAPI_ACCOUNT_EMAIL (Flow account) |
seedance-direct | Seedance 2.0 — free through your Higgsfield account with the engine that ships with videoclaw, or paid on the Ark/xskill API when that is chosen | Stylized/artistic/product; Asset-Library identity | The bundled Higgsfield engine set up against an eligible account, or VCLAW_SEEDANCE_DIRECT_NATIVE=1 + SUTUI_API_KEY for the paid API (neither ⇒ the route refuses rather than billing) |
runway-useapi | Runway, through your Runway account (seedance-2, gen4.x, kling, veo, sora, wan…) | Edit-heavy, audio-aware; explore mode depends on account entitlement | USEAPI_API_TOKEN + account configuration (USEAPI_ACCOUNT_EMAIL) |
dreamina-useapi | Seedance 2.0, through a Dreamina (CapCut) account — registered, not pursued since 2026-09-09 | Keyframe i2v, 1080p/4k on CA accounts (use seedance-direct instead) | USEAPI_API_TOKEN + VCLAW_DREAMINA_ACCOUNT |
magnific-rest | Magnific, through your Magnific/Freepik account | Animating a still: MiniMax Live / PixVerse / Runway / Kling / LTX | MAGNIFIC_API_KEY |
seedance-modelark | Seedance 2.5 (or 2.0 fast / mini), the official Dreamina API on BytePlus ModelArk, billed per second to your BytePlus account (a 4 s 720p clip billed 87,300 tokens ≈ USD 0.93 on 2026-09-21); never chosen by default | Up to 30 reference images + 10 videos + 10 audio clips, 4–30 s clips, native audio; first/last-frame i2v | ARK_API_KEY (optional VCLAW_MODELARK_MODEL, VCLAW_MODELARK_BASE_URL) |
reapi-seedance | Seedance 2.5 "Less Restriction" (content filter off), served by reAPI — through the treg catalog on your treg token, or with your own reAPI key. Paid per second of output; opt-in only | A photograph of a real person as the subject AND a voice clip as the speech reference in one render, with native speech/SFX/music; 4–30 s; 480p/720p/1080p | VCLAW_REAPI_SEEDANCE_VIA=treg + TREG_TOKEN, or VCLAW_REAPI_SEEDANCE_VIA=direct + REAPI_API_KEY + GO_BANANAS_API_KEY (hosts the references) |
Identity-lock differs per route (this is load-bearing): Google Veo (Flow) locks on a character you registered in Flow; Seedance/Runway/Dreamina lock on reference sheets + a rich per-subject visual descriptor (never a bare name, never a raw photoreal face — those trip real-person content filters). Describe characters by visual descriptor in prompts. reapi-seedance is the API-keyed exception: with its content filter off, a real photograph IS accepted as the subject reference and a voice clip as the speech reference, so a self-portrait talking clip can be one render there (the free Higgsfield engine behind seedance-direct does the same through a browser session). It is paid per second and never chosen for you.
No silent fallback across routes — a route never quietly switches to another materially different path (ADR 0001). Prompting is per-model; do not reuse one prompt style across routes.
Cinema queue routes (a separate id space)
The durable Cinema queue (vclaw video cinema-work) routes to its own provider-neutral route ids. These are not interchangeable with the core route ids above and are never merged with them: each declares its own transport and fallbackPolicy: forbidden, which is what keeps ADR 0001 enforceable.
| Route id | Transport | Account class | Media | What it is |
|---|---|---|---|---|
gobananas-images | gobananas-subscription-images | subscription-unlimited | image | Preferred subscription image route for identity and controlled reference work. |
openai-images | openai-image-api | api-credits | image | Provider-neutral alternative image route; selection is explicit and never automatic after failure. |
higgsfield-images | official-higgsfield-cli-images | provider-credits | image | Optional Higgsfield image route for provider-specific image workflows. |
higgsfield-unlimited | higgs-cloak-unlimited | subscription-unlimited | video | Signed-in unlimited-account transport, measured at one render at a time. |
higgsfield-cli | official-higgsfield-cli | provider-credits | video | Official CLI route for premium models and specialist cinematic workflows. |
seedance-xskill | xskill-ark | api-credits | video | Explicit paid xskill/ARK route; legacy seedance-direct is a compatibility alias outside Cinema. |
Audio and finish backends
Audio commands (narrate, dialogue, sfx, soundtrack) and the finishing pass (finish) select a backend by id. Every listed env var is a name that can satisfy the backend's gate — where several are listed for one backend, any one of them is enough (they are alternative sources of the same credential).
| Backend id | Kind | Name | Env that satisfies it |
|---|---|---|---|
suno | music | Suno (Kie.ai) | KIE_API_KEY |
lyria | music | Lyria (Vertex AI) | GOOGLE_CLOUD_PROJECT / VERTEX_PROJECT / GCLOUD_PROJECT |
lyria3 | music | Lyria 3 (Gemini API) | GEMINI_API_KEYS / GOOGLE_API_KEYS / GOOGLE_API_KEY |
flowmusic | music | FlowMusic (Lyria 3 Pro, useapi.net) | USEAPI_API_TOKEN |
mureka | music | Mureka songs and instrumentals (useapi.net) | USEAPI_API_TOKEN; linked Mureka account |
gemini-tts | tts | Gemini TTS | GEMINI_API_KEYS / GOOGLE_API_KEYS / GOOGLE_API_KEY |
elevenlabs-tts | tts | ElevenLabs TTS | ELEVENLABS_API_KEY |
mureka-tts | tts | Mureka narration and dialogue (useapi.net) | USEAPI_API_TOKEN; numeric speech voice ID |
nari-tts | tts | Nari Labs narration and dialogue, WAV | NARI_API_KEY; optional NARI_TTS_MODEL |
elevenlabs-sfx | sfx | ElevenLabs Sound Effects | ELEVENLABS_API_KEY |
Finish backends (vclaw video finish --backend <id>): topaz-starlight, topaz-proteus, topaz-gaia, magnific-precision, runway-topaz-free, ffmpeg-upscale, realesrgan-x4plus, topaz-local, google-flow. ffmpeg-upscale is the free, local one: ffmpeg only, no key and no install, providerCalls: 0.
Every id on this page is derived from the code that declares it — the route prerequisites in src/video/provider-platform/route-prerequisites.ts, composed into a manifest by src/video/capability-contract.ts. capability-contract.test.ts fails when a table here stops listing every id.
4. The production pipeline (canonical stage order)
init → brief → storyboard → assets → review → publish
│
readiness → plan → queue → quote → authorise → cinema-work → review
└→ direct produce/execute → execute-statusinit <slug>createsprojects/<slug>/under the workspace root (~/videoclawby default; override--root/VCLAW_WORKSPACE).briefsets intent + execution profile (aspect/quality/resolution/audio/veo-model).storyboarddefines scenes (--scene …or--template).assetsbinds per-scene keyframes / references.planpicks the route and builds an execution plan. Plainproduceis direct live execution unless--dry-runis set;produce --auto-chaincompiles a queue instead. See the spend model below.review/publishare the approval + delivery stages. For explicit film plans, current full-playback edit evidence is required; storyboard approval and sampled-frame QC do not replace it. See Shared filmmaking workflow for contracts and route limitations.
Two production modes (--mode storyboard|director): director mode adds a storyboard-approval gate before any provider spend.
Higher-level front doors (recommended over raw stages for new work):
vclaw studio <goal>— plan-only-by-default planning layer; prints the exactvclaw video …commands for 11 goals (create-video,creator-demo,copy-reference,presenter-video,music-video,ugc-campaign,existing-project,review-regenerate,publish-deliver,brand-campaign,character-video).--executeruns the plan (provably dry by default;--confirm-spendto render).- Concierge / VideoClaw (
skills/concierge) — guided menu, idea→video, plan→preview→spend order.
5. Spend & safety model (READ THIS before invoking anything paid)
There is no universal dry-run or spend flag. Read vclaw schema --json and the command's reference before acting. The execution families differ:
| Family | Default / preview | Live boundary |
|---|---|---|
produce --auto-chain | Compiles a durable continuity queue; provider calls: zero | Rejects --dry-run, --execute and --confirm-spend; run returned tasks with cinema-work |
pool | Compiles independent queued tasks; --dry-run previews without writing | --execute retired; enqueue rejects --confirm-spend |
batch-submit --project <slug> | Compiles manifest jobs into the durable queue | Immediate --execute retired; new jobs run with cinema-work |
cinema-work-quote / cinema-authorize | Exact provider quote, then persisted user authorisation bound to its hash | Quotation can contact the provider, but does not submit a render |
cinema-work | Acts on the saved task; reconciles an existing submitted job | Paid-path submission needs exact quote/hash, authorisation, quote adapter and --confirm-spend; Runway explore needs --confirm-provider-call on first submission |
Plain produce / execute | Live by default; --dry-run previews | Uses direct execution readiness and director approval when applicable; do not assume a standalone --confirm-spend gate |
| Image/audio/hosted finish commands | Many support --dry-run; defaults vary by command | Follow their individual --confirm-spend, credentials and execution rules |
| Local media post-production | No generation-provider charge; some commands write/render by default | Requires local files/tools; a local render is not a provider call |
For exact queued Flow commands, see the queued Flow task example in the CLI reference. A zero-credit quote still follows quote/authorisation checks. “Free” provider lanes mean no additional generation charge only when the configured account, subscription and selected operation qualify; external subscriptions, API services and availability are separate. Confirm current entitlement from the account.
Output is structured JSON for agent use. A render exit status does not establish content quality: inspect the output and retain review evidence before delivery.
6. Command orientation by function
Run vclaw schema --json for exact flags. Every registered command appears in exactly one group below (a test holds this map to the schema); the groups are orientation, the flags live in docs/CLI_REFERENCE.md.
Durable Cinema production and lane coordination
cinema-status · cinema-work-quote · cinema-authorize · cinema-work · cinema-discover · cinema-quote · cinema-execute · cinema-sync · lane (lane status) · planning & gates: cinema-create · cinema-migrate · cinema-history-import · cinema-approve · cinema-preflight · cinema-compile · review & delivery: cinema-console · cinema-console-live · cinema-ingest · cinema-review · cinema-promote · cinema-archive · cinema-restore · cinema-deliver (see the CLI reference for their distinct contracts).
The official Higgsfield CLI adapter is separate from the bootstrapped browser engine. New queued work uses immutable tasks and receipts; batch-monitor is only for historical submitted queues.
Project lifecycle & orchestration
init · brief · storyboard · assets · review · publish · create (one-shot create+hydrate) · auto · iterate · run-pipeline · approve · readiness · plan (alias execution-plan) · produce (alias execute) · execute-status · execute-cancel · execute-abandon · execute-bind · pool (N independent scenes) · render-scenes (route-fallback ladder) · set-execution-profile · set-meta · cost-estimate · archive-project · stock-search · stock-import (licensed stock via Pexels) · migrate-home (deprecated, one-shot) · import-legacy (deprecated, historical)
Provider / environment
providers · verify-env
Audio
narrate (TTS) · dialogue (multi-speaker) · sfx · soundtrack (music) · mureka (account status, voice IDs and submitted-job lookup) · voice-clone (drift-proof voice lock) · vocal-guides (free lip-sync guide exports)
Image / motion-graphics / media generation
gen-image (diegetic stills) · overlay (graphic/alert/lower-third) · motion-overlay (speech-synced reels) · mograph-sheet / mograph-pack / mograph-render / mograph-logos (style-locked motion graphics — see docs/MOGRAPH.md) · music-video · stitch-ad · title-card · storyboard-grid (shot-spec sheet) · outpaint-keyframe
Prompt-craft & direction
multi-shot · filmmaking-prompts · prompt-lint · director-blueprint · director-preflight · brand-definition · brand-extract · cinema-profile · clone-plan
Characters, references & consistency
character-add · character-auto-create · environment-auto-create · character-import-library · character-list · character-show · character-consistency · consistency-audit · reference-sheet-add|list|show|bind|validate · seedance-register-assets (Asset Library identity lock) · cinema-image-plan · cinema-image-compile · cinema-image-quote · cinema-image-ingest · cinema-image-review (hash-bound reference preparation, queue compilation, exact quote and review)
Cartoon-show layer
show-bible (world index) · show-preflight (route-aware readiness gate)
Scene candidates & chaining
candidates-list|show · select-candidate · reject-candidate · reroll-scene · chain-from · unchain · candidates-migrate-from-assets (deprecated, one-shot)
Assemble, finish & local post-production
assemble (stitch; --from-clips) · finish (upscale/HD/4K) · image-ops (still upscale) · lipsync · diagnose (output-quality troubleshoot) · animation-styles · remix-narrated · make-vertical · make-square · make-loop · thumbnail · burn-subtitles · verify-final · qc · motion-qc · clip-qc · keyframe-qc · match-highlights (event-only reels from a long fixed-camera recording, via Gemini agentic video understanding; --events-file … --place buys only the timing for an event list the free match-highlights-local skill already found)
Review & delivery portals
review-ui (interactive editor) · review-autopilot · storyboard-review · portal · portal-index · publish-preview · publish-portal-index · publish-metadata · publish-package (local upload package, never uploads) · creator-ui (loopback product shell) · creator-demo (zero-key demo project)
Flow (Veo) specific
flow-r2v · flow-register-characters · flow-register-voices · veo status|list|history|resume|reset|cancel · veo useapi:accounts|captcha|health|image|image:upscale|gif|upscale
Overnight batch queue
batch-submit · batch-monitor (deprecated → cinema-sync) · batch-status (deprecated → cinema-status)
Reference-ad cloning
clone-init · clone-execute (clone-ad is a deprecated spelling) · storyboard-from-clone · storyboard-still-add
Analysis
analyze (analyze-template is a deprecated spelling)
Ops, reporting & portfolio
list · index · monitor (localhost cockpit) · status · report · report-snapshot|history|diff · metrics · trends · next-actions · workload · dependencies · doctor-project · doctor-portfolio · artifact-history · export-csv · export-obsidian · sync-obsidian · scaffold-obsidian-vault
Templates, library & playbooks
template-list|show|save|create|validate · storyboard-template-list|show · list-library · find-library · library · prompt-lib-list|show · playbook-list|show
Top-level families (not video …)
vclaw studio <goal> (planning front door) · vclaw mcp serve (MCP server) · vclaw veo <verb> (Bun/Flow subprocess) · vclaw schema (this surface as JSON)
7. On-disk project model (the source of truth)
projects/<slug>/
project.json # manifest: slug, mode, state, execution profile
artifacts/ # canonical JSON: brief, storyboard, story-bible,
# asset-manifest, execution-plan, review-report, …
history/ # append-only artifact snapshots
audio/ # narration.mp3, soundtrack-*.mp3, dialogue-*, sfx-*
outputs/scene-<i>.mp4 # per-scene rendered clips
final/{videos,images,audio}/ # staged deliverables (portal reads these)
checkpoints/ # one per stage; tracks approval states
events/events.jsonl # append-only timeline
characters/ # character profiles + identity anchorsState lives on disk, not in memory — drive work from the artifacts, not from re-typed paths. schemas/video/* JSON Schemas are the source of truth for artifact shapes.
8. MCP (read-only agent access)
vclaw mcp serve exposes a stdio MCP server with read-only tools: list_projects, get_project_status, get_artifacts, get_event_log, list_provider_routes. Writes stay CLI-only by design.
9. Setup and support boundaries
| Capability layer | Prerequisites | Installation boundary |
|---|---|---|
| Core CLI, planning, artifacts, read-only MCP | Node >=20.10; Node dependencies; writable workspace | Available from a built source checkout or installed package |
| Local assembly and media checks | FFmpeg/ffprobe; fonts where used | External tools, not installed by npm |
| Flow/Veo sidecar | Bun >=1.3.5; sidecar dependencies; route credentials | Bundled source still needs bun install; copy to a writable location if the installed package cannot be modified |
| Python skills/helpers | Python 3.12+; workflow dependencies; sometimes FFmpeg/browser/provider access | Bundled helper source is not a preconfigured Python environment; external factories and personal presets need extra setup from you |
| Local/shared lane coordination | sqlite3 CLI locally; authenticated Cloudflare coordinator for shared slots | Shared service is separately deployed; a synced SQLite file is not a shared coordinator |
| Remote generation, TTS, image/vision, hosted finish | Selected route's credentials, provider access and account entitlement | Source availability and local tests do not prove live provider support |
| Review and delivery | Local authenticated review server; R2/Wrangler setup for uploads | Publishing a portal is separate from generating it locally |
- Workspace root:
~/videoclaw(override--root→VCLAW_WORKSPACE→VIDEOCLAW_WORKSPACE). Keep project data separate from application resources. - Credentials: configure only the selected route:
USEAPI_API_TOKENand account variables for Flow/Runway/Dreamina,SUTUI_API_KEYfor Seedance, Gemini/Google keys for applicable audio/vision,ELEVENLABS_API_KEY,MAGNIFIC_API_KEY, or image-provider keys as required by that command. - Start offline with
vclaw schema --json,vclaw video providers, or an explicitly documented plan/dry-run path. Do not append--dry-runblindly. - See the installation guide and Release Readiness for setup commands and the distinction between local, packaged and live evidence.
10. What videoclaw is NOT
- Not a general video editor / NLE — it's a pipeline over AI generators + FFmpeg.
- Not a hosted service — it's a local CLI operating on a local project tree.
- It does not guarantee provider availability, fixed pricing or perfect identity consistency. Provider moderation can reject inputs; review the selected route and current account constraints before authorising a render.