Skip to main content

Autonomous Direction

Nine tools that power the /direct pipeline. Run the whole thing in one call with generate_video, or compose it yourself: extract briefs from project context, compose storyboards as design checkpoints, plan story beats across multiple strategies, score and revise candidates, and run automated revision loops until convergence.


generate_video

One-shot video pipeline: a natural-language prompt in, a render-ready spec out. Runs the entire brief → generate → analyze → plan → compile → critique → evaluate flow in a single call and returns scenes, a sequence manifest, compiled timelines, and quality scores — everything Remotion needs to render. Reach for this when you want a result fast; use the granular tools below (or the /direct skill) when you want to inspect and steer each step.

  • Name
    prompt
    Type
    string
    Description

    Required. Natural language video description, e.g. "30-second promo for an AI finance dashboard, cinematic-dark, prestige style"

  • Name
    style
    Type
    string
    Description

    Override auto-detected style pack

  • Name
    personality
    Type
    string
    Description

    Override auto-detected personality

  • Name
    enhance
    Type
    boolean
    Description

    Enable LLM enhancement (requires ANTHROPIC_API_KEY). Default: false.

Returns: Scenes, sequence manifest, compiled per-scene timelines, and per-dimension quality scores — ready to hand to Remotion for rendering.

Try asking your AI:

"Generate a 20-second product launch video for our analytics dashboard"

"Make a montage-style sizzle reel from this prompt and score it"


extract_story_brief

Extract a structured story brief from project context. Parses the brief markdown, matches a sequence archetype, and infers personality/style from the brand. Returns audience, promise, tone, features, proof points, closing beat, and narrative template.

  • Name
    project
    Type
    object
    Description

    Contents of project.json for the target project.

  • Name
    brief
    Type
    string
    Description

    Brief markdown text.

  • Name
    storyboard
    Type
    object
    Description

    Storyboard JSON. Optional.

  • Name
    scenes
    Type
    object[]
    Description

    Scene definitions. Optional — improves inference when supplied.

  • Name
    brand
    Type
    object
    Description

    Brand package. Optional.

  • Name
    overrides
    Type
    object
    Description

    Explicit field overrides that take precedence over inference.

Returns: Structured story brief with audience, promise, tone, features, proof points, closing beat, matched archetype, and narrative template — plus quality warnings.

Try asking your AI:

"Extract a story brief from the fintech-demo project"


compose_storyboard

Compose a storyboard — panels with intent, content, visual_direction, and motion_notes — from a story brief and archetype. Wedges between extract_story_brief and plan_story_beats as the design checkpoint. Returns the storyboard JSON shape defined in docs/cinematography/specs/storyboard-format.md. With ANTHROPIC_API_KEY set and options.enhance left at its default, an LLM enriches visual_direction with specific pixel, weight, and color values; otherwise the tool returns a deterministic skeleton.

  • Name
    story_brief
    Type
    object
    Description

    Required. Output of extract_story_brief. Required.

  • Name
    brief
    Type
    string
    Description

    Raw brief markdown text. Optional — feeds LLM enrichment context.

  • Name
    brand
    Type
    object
    Description

    Brand package (output of get_brand_package). Optional but strongly recommended.

  • Name
    project
    Type
    object
    Description

    project.json contents. Used for title, tagline, disclaimer.

  • Name
    archetype_slug
    Type
    string
    Description

    Override the archetype. Defaults to story_brief.narrative_template.

  • Name
    options
    Type
    object
    Description

    Composition options.

Returns: A storyboard JSON object with storyboard_id, archetype reference, and an array of panels — each panel carrying intent, content, visual_direction, and motion_notes ready for plan_story_beats to consume.

Try asking your AI:

"Compose a storyboard for the fintech-demo brief using the feature-reveal archetype"


plan_story_beats

Map a story brief onto a sequence archetype to produce a concrete beat plan with durations, camera intents, transitions, and continuity opportunities. Optionally snaps to audio beats.

  • Name
    story_brief
    Type
    object
    Description

    Required. Output of extract_story_brief.

  • Name
    archetype_slug
    Type
    string
    Description

    Required. Sequence archetype slug (e.g., brand-teaser, feature-reveal, onboarding-explainer).

  • Name
    storyboard
    Type
    object
    Description

    Output of compose_storyboard. When provided, panel-level visual_direction + motion_notes override archetype defaults per beat.

  • Name
    audio_beats
    Type
    object
    Description

    Beat data from analyze_beats. When provided, beat durations snap to audio beats.

  • Name
    options
    Type
    object
    Description

    Planning options. duration_target_s (number) sets total target duration; strategy picks pacing bias — tight, loose, or cinematic; personality overrides the personality used to source choreography companion primitives.

Returns: A beat plan with per-scene durations, camera intents, transition types, and continuity opportunities, ready to drive plan_sequence or instantiate_sequence_archetype.

Try asking your AI:

"Plan story beats for this brief using the feature-reveal archetype with the cinematic strategy"


score_candidate_video

Unified 6-dimension scorecard for a candidate video. Runs all existing evaluators (sequence eval, per-scene critic, motion density, brand compliance, product clarity, audio sync) and returns a unified 0-1 score card with findings and revision recommendations.

The six dimensions:

  • hook — Opening impact and attention capture
  • narrative_arc — Story structure and progression
  • clarity — Message clarity
  • visual_hierarchy — Compositional quality per scene
  • motion_quality — Animation smoothness and appropriateness
  • brand_finish — Brand consistency and polish
  • Name
    manifest
    Type
    object
    Description

    Required. Sequence manifest to score.

  • Name
    scenes
    Type
    object[]
    Description

    Required. Scene definitions matching the manifest.

  • Name
    style
    Type
    string
    Description

    Style pack name. Used by the sequence evaluator.

  • Name
    brand
    Type
    object
    Description

    Brand package. Enables brand-finish scoring.

  • Name
    audio_beats
    Type
    object
    Description

    Beat data from analyze_beats. Enables audio-sync scoring.

  • Name
    weights
    Type
    object
    Description

    Custom per-dimension weights to override the default scorecard balance.

Returns: Unified 0-1 score card with overall and per-dimension scores, per-scene subscores, findings, and revision recommendations.

Try asking your AI:

"Score this candidate manifest and its scenes against the prestige style pack"


revise_candidate_video

Apply bounded revision operations to a manifest. Nine ops are available: trim, extend_hold, swap_transition, reorder, boost_hierarchy, compress, add_continuity, adjust_density, and more. Returns the revised manifest, revised scenes, and a diff log.

  • Name
    manifest
    Type
    object
    Description

    Required. Sequence manifest to revise.

  • Name
    scenes
    Type
    object[]
    Description

    Scene definitions that accompany the manifest.

  • Name
    revisions
    Type
    object[]
    Description

    Required. Array of revision ops shaped as { op, target, ...params }.

Returns: Revised manifest, revised scenes, and a diff log showing what each op changed.

Try asking your AI:

"Apply a trim op to scene 3 and an extend_hold op to scene 5"


compare_candidate_videos

Rank 2-3 scored video candidates. Returns overall rankings, per-dimension winners, and a recommendation with rationale and trade-off analysis.

  • Name
    candidates
    Type
    object[]
    Description

    Required. Array of candidate objects, each shaped as { candidate_id, strategy, score_card, manifest }.

Returns: Ranked candidates with overall ordering, per-dimension winners, trade-off analysis, and a recommended winner with rationale.

Try asking your AI:

"Compare these 3 candidate videos and recommend the best one"


auto_revise_loop

Autonomous revision loop: scores per-scene, picks the worst scenes, applies targeted revisions, re-scores, and repeats until convergence or max rounds. Returns the best manifest with full revision history.

  • Name
    manifest
    Type
    object
    Description

    Required. Starting sequence manifest.

  • Name
    scenes
    Type
    object[]
    Description

    Required. Scene definitions matching the manifest.

  • Name
    style
    Type
    string
    Description

    Style pack name. Used by the internal scorer.

  • Name
    brand
    Type
    object
    Description

    Brand package. Enables brand-finish scoring during each round.

  • Name
    audio_beats
    Type
    object
    Description

    Beat data from analyze_beats. Enables audio-sync scoring.

  • Name
    max_rounds
    Type
    number
    Description

    Maximum revision rounds. Default: 3.

  • Name
    min_improvement
    Type
    number
    Description

    Convergence threshold — stop when round-over-round score gain drops below this. Default: 0.01.

Returns: The best manifest discovered, the revised scenes, per-round score history, and a full revision log.

Try asking your AI:

"Run the auto-revise loop on this manifest for up to 5 rounds"


generate_brief_stub

Generate a brief markdown stub from project context. Pre-fills audience, promise, tone, features, and proof sections with inferred content so the author starts from a structured template instead of a blank page.

  • Name
    project
    Type
    object
    Description

    Contents of project.json for the target project.

  • Name
    scenes
    Type
    object[]
    Description

    Scene definitions. Optional — improves inference when supplied.

  • Name
    brand
    Type
    object
    Description

    Brand package. Optional.

Returns: A brief markdown stub with pre-filled audience, promise, tone, features, and proof sections.

Try asking your AI:

"Generate a brief stub for this project"

Was this page helpful?