Autonomous Direction
Nine tools that power the /direct pipeline. Run the whole thing in one call with generate_video, or compose it yourself: extract briefs from project context, compose storyboards as design checkpoints, plan story beats across multiple strategies, score and revise candidates, and run automated revision loops until convergence.
generate_video
One-shot video pipeline: a natural-language prompt in, a render-ready spec out. Runs the entire brief → generate → analyze → plan → compile → critique → evaluate flow in a single call and returns scenes, a sequence manifest, compiled timelines, and quality scores — everything Remotion needs to render. Reach for this when you want a result fast; use the granular tools below (or the /direct skill) when you want to inspect and steer each step.
- Name
prompt- Type
- string
- Description
Required. Natural language video description, e.g. "30-second promo for an AI finance dashboard, cinematic-dark, prestige style"
- Name
style- Type
- string
- Description
Override auto-detected style pack
- Name
personality- Type
- string
- Description
Override auto-detected personality
- Name
enhance- Type
- boolean
- Description
Enable LLM enhancement (requires ANTHROPIC_API_KEY). Default: false.
Returns: Scenes, sequence manifest, compiled per-scene timelines, and per-dimension quality scores — ready to hand to Remotion for rendering.
Try asking your AI:
"Generate a 20-second product launch video for our analytics dashboard"
"Make a montage-style sizzle reel from this prompt and score it"
extract_story_brief
Extract a structured story brief from project context. Parses the brief markdown, matches a sequence archetype, and infers personality/style from the brand. Returns audience, promise, tone, features, proof points, closing beat, and narrative template.
- Name
project- Type
- object
- Description
Contents of
project.jsonfor the target project.
- Name
brief- Type
- string
- Description
Brief markdown text.
- Name
storyboard- Type
- object
- Description
Storyboard JSON. Optional.
- Name
scenes- Type
- object[]
- Description
Scene definitions. Optional — improves inference when supplied.
- Name
brand- Type
- object
- Description
Brand package. Optional.
- Name
overrides- Type
- object
- Description
Explicit field overrides that take precedence over inference.
Returns: Structured story brief with audience, promise, tone, features, proof points, closing beat, matched archetype, and narrative template — plus quality warnings.
Try asking your AI:
"Extract a story brief from the fintech-demo project"
compose_storyboard
Compose a storyboard — panels with intent, content, visual_direction, and motion_notes — from a story brief and archetype. Wedges between extract_story_brief and plan_story_beats as the design checkpoint. Returns the storyboard JSON shape defined in docs/cinematography/specs/storyboard-format.md. With ANTHROPIC_API_KEY set and options.enhance left at its default, an LLM enriches visual_direction with specific pixel, weight, and color values; otherwise the tool returns a deterministic skeleton.
- Name
story_brief- Type
- object
- Description
Required. Output of
extract_story_brief. Required.
- Name
brief- Type
- string
- Description
Raw brief markdown text. Optional — feeds LLM enrichment context.
- Name
brand- Type
- object
- Description
Brand package (output of
get_brand_package). Optional but strongly recommended.
- Name
project- Type
- object
- Description
project.json contents. Used for title, tagline, disclaimer.
- Name
archetype_slug- Type
- string
- Description
Override the archetype. Defaults to
story_brief.narrative_template.
- Name
options- Type
- object
- Description
Composition options.
Returns: A storyboard JSON object with storyboard_id, archetype reference, and an array of panels — each panel carrying intent, content, visual_direction, and motion_notes ready for plan_story_beats to consume.
Try asking your AI:
"Compose a storyboard for the fintech-demo brief using the feature-reveal archetype"
plan_story_beats
Map a story brief onto a sequence archetype to produce a concrete beat plan with durations, camera intents, transitions, and continuity opportunities. Optionally snaps to audio beats.
- Name
story_brief- Type
- object
- Description
Required. Output of
extract_story_brief.
- Name
archetype_slug- Type
- string
- Description
Required. Sequence archetype slug (e.g.,
brand-teaser,feature-reveal,onboarding-explainer).
- Name
storyboard- Type
- object
- Description
Output of
compose_storyboard. When provided, panel-level visual_direction + motion_notes override archetype defaults per beat.
- Name
audio_beats- Type
- object
- Description
Beat data from
analyze_beats. When provided, beat durations snap to audio beats.
- Name
options- Type
- object
- Description
Planning options.
duration_target_s(number) sets total target duration;strategypicks pacing bias —tight,loose, orcinematic;personalityoverrides the personality used to source choreography companion primitives.
Returns: A beat plan with per-scene durations, camera intents, transition types, and continuity opportunities, ready to drive plan_sequence or instantiate_sequence_archetype.
Try asking your AI:
"Plan story beats for this brief using the feature-reveal archetype with the cinematic strategy"
score_candidate_video
Unified 6-dimension scorecard for a candidate video. Runs all existing evaluators (sequence eval, per-scene critic, motion density, brand compliance, product clarity, audio sync) and returns a unified 0-1 score card with findings and revision recommendations.
The six dimensions:
- hook — Opening impact and attention capture
- narrative_arc — Story structure and progression
- clarity — Message clarity
- visual_hierarchy — Compositional quality per scene
- motion_quality — Animation smoothness and appropriateness
- brand_finish — Brand consistency and polish
- Name
manifest- Type
- object
- Description
Required. Sequence manifest to score.
- Name
scenes- Type
- object[]
- Description
Required. Scene definitions matching the manifest.
- Name
style- Type
- string
- Description
Style pack name. Used by the sequence evaluator.
- Name
brand- Type
- object
- Description
Brand package. Enables brand-finish scoring.
- Name
audio_beats- Type
- object
- Description
Beat data from
analyze_beats. Enables audio-sync scoring.
- Name
weights- Type
- object
- Description
Custom per-dimension weights to override the default scorecard balance.
Returns: Unified 0-1 score card with overall and per-dimension scores, per-scene subscores, findings, and revision recommendations.
Try asking your AI:
"Score this candidate manifest and its scenes against the prestige style pack"
revise_candidate_video
Apply bounded revision operations to a manifest. Nine ops are available: trim, extend_hold, swap_transition, reorder, boost_hierarchy, compress, add_continuity, adjust_density, and more. Returns the revised manifest, revised scenes, and a diff log.
- Name
manifest- Type
- object
- Description
Required. Sequence manifest to revise.
- Name
scenes- Type
- object[]
- Description
Scene definitions that accompany the manifest.
- Name
revisions- Type
- object[]
- Description
Required. Array of revision ops shaped as
{ op, target, ...params }.
Returns: Revised manifest, revised scenes, and a diff log showing what each op changed.
Try asking your AI:
"Apply a trim op to scene 3 and an extend_hold op to scene 5"
compare_candidate_videos
Rank 2-3 scored video candidates. Returns overall rankings, per-dimension winners, and a recommendation with rationale and trade-off analysis.
- Name
candidates- Type
- object[]
- Description
Required. Array of candidate objects, each shaped as
{ candidate_id, strategy, score_card, manifest }.
Returns: Ranked candidates with overall ordering, per-dimension winners, trade-off analysis, and a recommended winner with rationale.
Try asking your AI:
"Compare these 3 candidate videos and recommend the best one"
auto_revise_loop
Autonomous revision loop: scores per-scene, picks the worst scenes, applies targeted revisions, re-scores, and repeats until convergence or max rounds. Returns the best manifest with full revision history.
- Name
manifest- Type
- object
- Description
Required. Starting sequence manifest.
- Name
scenes- Type
- object[]
- Description
Required. Scene definitions matching the manifest.
- Name
style- Type
- string
- Description
Style pack name. Used by the internal scorer.
- Name
brand- Type
- object
- Description
Brand package. Enables brand-finish scoring during each round.
- Name
audio_beats- Type
- object
- Description
Beat data from
analyze_beats. Enables audio-sync scoring.
- Name
max_rounds- Type
- number
- Description
Maximum revision rounds. Default:
3.
- Name
min_improvement- Type
- number
- Description
Convergence threshold — stop when round-over-round score gain drops below this. Default:
0.01.
Returns: The best manifest discovered, the revised scenes, per-round score history, and a full revision log.
Try asking your AI:
"Run the auto-revise loop on this manifest for up to 5 rounds"
generate_brief_stub
Generate a brief markdown stub from project context. Pre-fills audience, promise, tone, features, and proof sections with inferred content so the author starts from a structured template instead of a blank page.
- Name
project- Type
- object
- Description
Contents of
project.jsonfor the target project.
- Name
scenes- Type
- object[]
- Description
Scene definitions. Optional — improves inference when supplied.
- Name
brand- Type
- object
- Description
Brand package. Optional.
Returns: A brief markdown stub with pre-filled audience, promise, tone, features, and proof sections.
Try asking your AI:
"Generate a brief stub for this project"