concept-to-video

Concept to Video

Creates animated explainer videos from concepts using Manim (Python) as a programmatic animation engine.

Reference Files

File	Purpose
references/rules/pipeline-flow.md	RAG, ETL, CI/CD — sequential stage animations with arrows
references/rules/architecture-layers.md	System stacks, network layers, abstraction hierarchies
references/rules/algorithm-stepthrough.md	Sorting, search, graph traversal — stateful step-by-step animations
references/rules/comparison.md	Side-by-side A vs B, before/after, trade-off visualizations
references/rules/agent-interaction.md	Multi-agent message passing, distributed systems, pub/sub
references/rules/math-concept.md	Equations, formulas, geometric proofs — LaTeX-free by default
references/rules/training-loop.md	Gradient descent, RL loops, cyclic iterative processes
references/rules/transitions.md	Fade and wipe transitions between scene sections
references/rules/text-animation.md	Text replacement, progressive bullet reveal, callouts, emphasis
references/rules/layout.md	Canvas coordinates, VGroup arrangement, spacing guidelines
references/rules/audio-overlay.md	ffmpeg audio overlay — background music, voiceover, multi-track mixing
references/rules/voiceover-scaffold.md	Timing script generation, TTS handoff, narration best practices
references/rules/images.md	ImageMobject usage, logo/screenshot patterns, scaling and positioning
references/rules/subtitles.md	SRT generation from scene timing, ffmpeg subtitle burning
references/rules/multi-scene.md	Multiple Scene classes, ffmpeg concat, chapter-based composition
references/templates/data_flow_template.py	Parametric pipeline/data flow animation (config-driven STAGES list)
references/templates/comparison_template.py	Parametric side-by-side comparison (config-driven LEFT/RIGHT items)
references/templates/timeline_template.py	Parametric timeline animation (config-driven EVENTS list)
scripts/render_video.py	Wrapper around Manim CLI — handles quality, format, output path cleanup
scripts/add_audio.py	ffmpeg wrapper — audio overlay, volume, fade-in/out, trim-to-video

Why Manim as the engine

Manim is the "SVG of video" — you write Python code that describes animations declaratively, and it renders to MP4/GIF at any resolution. The Python scene file IS the editable intermediate: the user can see the code, request changes ("make the arrows red", "add a third step", "slow down the transition"), and only do a final high-quality render once satisfied. This makes the workflow iterative and controllable, exactly like concept-to-image uses HTML as an intermediate.

Workflow

Concept → Manim scene (.py) → Preview (low-quality) → Iterate → Final render (MP4/GIF)

1. Interpret the user's concept — determine the best animation approach
2. Design a self-contained Manim scene file — one file, one Scene class
3. Preview by rendering at low quality (-ql) for fast iteration
4. Iterate on the scene based on user feedback
5. Export final video at high quality using scripts/render_video.py

Step 0: Ensure dependencies

Before writing any scene, ensure Manim is installed:

# System deps (usually pre-installed)
apt-get install -y libpango1.0-dev libcairo2-dev ffmpeg 2>/dev/null

# Python package
pip install manim --break-system-packages -q

Verify with: python3 -c "import manim; print(manim.__version__)"

Step 1: Interpret the concept

Determine the best animation pattern, then read the matching rule file before writing any code.

User intent	Rule file to read	Key Manim primitives
Explain a pipeline/flow	references/rules/pipeline-flow.md	Arrow, Rectangle, Text, AnimationGroup
Show architecture layers	references/rules/architecture-layers.md	VGroup, Arrange, FadeIn with shift
Algorithm step-through	references/rules/algorithm-stepthrough.md	Transform, ReplacementTransform, Indicate
Compare approaches	references/rules/comparison.md	Split screen VGroups, simultaneous animations
Mathematical concept	references/rules/math-concept.md	MathTex, geometric shapes, Rotate, Scale
Agent/multi-system interaction	references/rules/agent-interaction.md	Arrows between entities, Create/FadeOut
Training/optimization loop	references/rules/training-loop.md	Loop with Transform, ValueTracker, plots
Timeline/history	references/templates/timeline_template.py	NumberLine, sequential Indicate
Embed images or screenshots	references/rules/images.md	ImageMobject, SVGMobject
Add subtitles or captions	references/rules/subtitles.md	SRT generation, ffmpeg subtitle burn
Multiple distinct chapters	references/rules/multi-scene.md	Multiple Scene classes, ffmpeg concat
Add audio or voiceover	references/rules/audio-overlay.md	ffmpeg, scripts/add_audio.py
Transition between sections	references/rules/transitions.md	FadeOut all, shift off-screen
Text reveal, callouts, emphasis	references/rules/text-animation.md	ReplacementTransform, LaggedStart, Indicate
Positioning, spacing, layout	references/rules/layout.md	next_to, arrange, to_edge, move_to

Step 2: Design the Manim scene

Template-first vs from-scratch

Check whether a parametric template covers the concept before writing a scene from scratch:

If the concept is...	Start with template
A linear pipeline (A→B→C→D)	references/templates/data_flow_template.py — edit STAGES
A two-option comparison	references/templates/comparison_template.py — edit LEFT_ITEMS, RIGHT_ITEMS
A chronological timeline	references/templates/timeline_template.py — edit EVENTS
Anything else	Write from scratch using the relevant rule file

When using a template: copy it to the working directory, edit the config constants at the top, do not restructure the class.

Core rules:

• Single file, single Scene class: Everything in one .py file with one class XxxScene(Scene).
• Self-contained: No external assets unless absolutely necessary. Use Manim primitives for everything.
• Readable code: The scene file IS the user's artifact. Use clear variable names, comments for each animation beat.
• Color with intention: Use Manim's color constants (BLUE, RED, GREEN, YELLOW, etc.) or hex colors. Max 4-5 colors. Every color should encode meaning.
• Pacing: Include self.wait() calls between logical sections. 0.5s for breathing room, 1-2s for major transitions.
• Text legibility: Use font_size=36 minimum for body text, font_size=48+ for titles. Test at target resolution.
• Scene dimensions: Default Manim canvas is 14.2 × 8 units (16:9). Keep content within ±6 horizontal, ±3.5 vertical.

Animation best practices

# DO: Use animation groups for simultaneous effects
self.play(FadeIn(box), Write(label), run_time=1)

# DO: Use .animate syntax for property changes
self.play(box.animate.shift(RIGHT * 2).set_color(GREEN))

# DO: Stagger related elements
self.play(LaggedStart(*[FadeIn(item) for item in items], lag_ratio=0.2))

# DON'T: Add/remove without animation (jarring)
self.add(box)  # Only for setup before first frame

# DON'T: Make animations too fast
self.play(Transform(a, b), run_time=0.3)  # Too fast to read

Structure template

from manim import *

class ConceptScene(Scene):
    def construct(self):
        # === Section 1: Title / Setup ===
        title = Text("Concept Name", font_size=56, weight=BOLD)
        self.play(Write(title))
        self.wait(1)
        self.play(FadeOut(title))

        # === Section 2: Core animation ===
        # ... main content here ...

        # === Section 3: Summary / Conclusion ===
        # ... wrap-up animation ...
        self.wait(2)

Step 3: Preview render

Use low quality for fast iteration:

python3 scripts/render_video.py scene.py ConceptScene --quality low --format mp4

This renders at 480p/15fps — fast enough for previewing timing and layout. Present the video to the user.

Step 4: Iterate

Common refinement requests and how to handle them:

Request	Action
"Slower/faster"	Adjust run_time= params and self.wait() durations
"Change colors"	Update color constants
"Add a step"	Insert new animation block between sections
"Reorder"	Move code blocks around
"Different layout"	Adjust .shift(), .next_to(), .arrange() calls
"Add labels/annotations"	Add Text or MathTex objects with .next_to()
"Make it loop"	Add matching intro/outro states

Step 5: Final export

Once the user is satisfied:

python3 scripts/render_video.py scene.py ConceptScene --quality high --format mp4

Quality presets

Preset	Resolution	FPS	Flag	Use case
low	480p	15	-ql	Fast preview
medium	720p	30	-qm	Draft review
high	1080p	60	-qh	Final delivery
4k	2160p	60	-qk	Presentation quality

Format options

Format	Flag	Use case
mp4	--format mp4	Standard video delivery
gif	--format gif	Embeddable in docs, social
webm	--format webm	Web-optimized

Delivering the output

Present both:

1. The .py scene file (for future editing)
2. The rendered video file (final output)

Copy the final video to /mnt/user-data/outputs/ and present it.

Step 5.5: Optional audio overlay

If the user provides audio (music or voiceover), or requests it:

# Background music at 25% volume with fade-in/out
python3 scripts/add_audio.py final.mp4 music.mp3 \
    --output final_with_audio.mp4 \
    --volume 0.25 --fade-in 2 --fade-out 3 --trim-to-video

# Voiceover at full volume, trimmed to video length
python3 scripts/add_audio.py final.mp4 voiceover.mp3 \
    --output final_narrated.mp4 --trim-to-video

For voiceover scripting before recording, read references/rules/voiceover-scaffold.md. For subtitles/captions, read references/rules/subtitles.md. For advanced multi-track mixing, read references/rules/audio-overlay.md.

Error Handling

Error	Cause	Resolution
ModuleNotFoundError: manim	Manim not installed	Run Step 0 setup commands
pangocairo build error	Missing system dev headers	apt-get install -y libpango1.0-dev
FileNotFoundError: ffmpeg	ffmpeg not installed	apt-get install -y ffmpeg
Scene class not found	Class name mismatch	Verify class name matches CLI argument
Overlapping objects	Positions not calculated	Use .next_to(), .arrange(), explicit .shift() calls
Text cut off	Text too large or positioned near edge	Reduce font_size or adjust position within ±6,±3.5
Slow render	Too many objects or complex transformations	Reduce object count, simplify paths, use lower quality
LaTeX Error	LaTeX not installed (for MathTex)	Use Text instead, or install texlive-latex-base

LaTeX fallback

If LaTeX is not available, avoid MathTex and Tex. Use Text with Unicode math symbols instead:

# Instead of: MathTex(r"\frac{1}{n} \sum_{i=1}^{n} x_i")
# Use:        Text("(1/n) Σ xᵢ", font_size=36)

Agentic Mode (Opt-In)

Single-shot mode (default) is fast and cheap — the coder writes scene.py directly from a concept. Use agentic mode for production-quality renders where layout correctness and asset resolution matter enough to justify additional LLM and VLM calls.

Pipeline

concept
  └─► plan_storyboard.py ──► storyboard.json
            │
            ▼
      fetch_assets.py (optional)
            │
            ▼
      coder writes scene.py
            │
            ▼
      render_video.py --max-fix-attempts N
            │  ▲
            │  └─ LLM fixup loop (on failure, up to N retries)
            ▼
      critic_pass.py --critic
            │  ▲
            │  └─ VLM layout patch (1 call with M image blocks)
            ▼
       final MP4

Flag Reference

Script	Flag	Default	Hard cap	Effect	Cost impact
render_video.py	--max-fix-attempts	0	3	LLM-assisted auto-fix on render failure; 0 = disabled	+1 LLM call per retry
critic_pass.py	--critic	disabled	—	Enable the VLM critic pass; noop without this flag	+1 VLM call (N image blocks)
critic_pass.py	--critic-budget	50000	—	Token budget for critic call; aborts loudly if exceeded	Sets ceiling; use to prevent runaway spend
critic_pass.py	--frames	5	10	Frames sampled from the rendered video for the critic	More frames → higher token cost per critic run
fetch_assets.py	--adapter	none	—	Asset backend: local, iconfinder, none	iconfinder adds external API calls
fetch_assets.py	--asset-dir	—	—	Root directory for --adapter=local; required with local	None

Cost Tradeoffs

The fixup loop adds one LLM call per failed render attempt — with --max-fix-attempts 3 you may pay up to 3 extra calls before the loop exhausts or succeeds. The critic pass adds one VLM call containing N PNG image blocks (default 5, max 10); each frame adds roughly 1 token per 800 bytes of base64-encoded PNG, so complex scenes at high resolution are materially more expensive. Setting --critic-budget to a conservative token ceiling (e.g. 20000) causes BudgetExceededError before the API call is made, so you never pay for an accidentally oversized request — the error is loud and non-recoverable by design.

Invocation Example

# 1. Plan
python3 scripts/plan_storyboard.py "explain transformer self-attention" \
    --output storyboard.json

# 2. (Optional) Fetch assets
python3 scripts/fetch_assets.py storyboard.json \
    --adapter local --asset-dir ./assets --output resolved.json

# 3. Coder writes scene.py (Claude writes this from storyboard.json)

# 4. Render with auto-fix
python3 scripts/render_video.py scene.py AttentionScene \
    --quality high --format mp4 --max-fix-attempts 3 \
    --output final.mp4

# 5. Critic pass
python3 scripts/critic_pass.py scene.py final.mp4 \
    --critic --critic-budget 40000 --frames 5

Agentic pipeline design (storyboard planner, auto-fix loop, VLM critic) is adapted from Code2Video (arXiv 2510.01174, MIT). Vendored prompt templates live in references/code2video/ alongside the upstream LICENSE. Full vendoring record, pinned commit, and re-sync policy are tracked in root ATTRIBUTIONS.md.

Limitations

• Manim + ffmpeg required — cannot render without these dependencies.
• Audio is post-render only — Manim renders silent MP4s. Use scripts/add_audio.py to overlay audio after export.
• LaTeX optional — MathTex requires a LaTeX installation. Fall back to Text with Unicode for math.
• Render time scales with complexity — a 30-second 1080p scene with many objects can take 1-2 minutes to render.
• 3D scenes require OpenGL — ThreeDScene may not work in headless containers. Stick to 2D Scene class.
• No interactivity — output is a static video file, not an interactive widget.
• GIF output is silent — audio overlay only works with MP4/WEBM output formats.

Design anti-patterns to avoid

• Walls of text on screen — keep to 3-5 words per label, max 2 lines
• Everything appearing at once — use staged animations with LaggedStart
• Uniform timing — vary run_time to create rhythm (fast for simple, slow for important)
• No visual hierarchy — use size, color, and position to guide attention
• Rainbow colors — 3-4 intentional colors max
• Ignoring the grid — align objects to consistent positions using arrange/align