comfyui-video-production

ComfyUI 视频制作流水线

使用 ComfyUI 进行端到端视频制作编排，支持自动错误恢复、质量验证和实例管理。

快速开始：选择哪种流水线？

要制作多镜头叙事视频？ → 关键帧流水线 - 生成关键帧 → 动画化 → 添加转场并拼接

要让现有图像动起来？ → I2V 批处理流水线 - 加载图像 → 排队处理 I2V 任务 → 自动验证 → 合并

需要场景之间流畅过渡？ → 转场流水线 - 通过 FFmpeg 添加交叉淡化、运动模糊和缩放效果

ComfyUI 卡住或崩溃了？ → 实例管理器 - 自动重启、健康检查、队列监控

要调试视频问题？ → 验证套件 - 检查分辨率、FPS、编解码器、人脸一致性和调色

---

核心流水线

流水线 1：从关键帧到视频（完整叙事）

适用场景： 制作包含多个不同镜头的故事类视频

1. Keyframe Generation Phase
   - Generate consistent keyframes with IP-Adapter/LoRA
   - Validate face consistency, lighting, pose progression
   - Save to organized directory structure
   - Auto-retry failed generations

2. I2V Animation Phase
   - Queue each keyframe to I2V model (Wan 2.2, LTX-2, AnimateDiff)
   - Monitor progress via ComfyUI API
   - Validate each clip (resolution, fps, duration)
   - Auto-retry with different seeds if failed

3. Concatenation Phase
   - Pre-flight validation (ensure all clips match)
   - Apply transition effects (crossfade, motion blur)
   - FFmpeg encoding with proper codec
   - Export final video with metadata

4. Quality Assurance
   - Face consistency check across clips
   - Color grading consistency
   - Audio sync validation (if applicable)
   - Generate QA report

预期输出： 带有流畅转场的连贯完整视频

---

流水线 2：批量处理 I2V

适用场景： 需要分别为多张图像制作动画

1. Image Discovery
   - Scan directory for source images
   - Validate image specs (resolution, format)
   - Generate processing manifest

2. Parallel I2V Queue
   - Queue all images to ComfyUI with appropriate prompts
   - Stagger submissions to avoid overload
   - Monitor queue depth and ETA

3. Progressive Validation
   - Check each completed video immediately
   - Flag issues (wrong resolution, fps, corruption)
   - Auto-retry flagged videos

4. Export & Organize
   - Move validated videos to output directory
   - Generate index with metadata
   - Create contact sheet (thumbnail preview grid)

预期输出： 经过验证的动画片段目录

---

流水线 3：添加转场并拼接视频

适用场景： 使用专业转场合并现有视频片段

1. Clip Validation
   - Verify all clips exist and are readable
   - Check resolution, fps, codec consistency
   - Report mismatches with fix suggestions

2. Transition Planning
   - Detect scene changes (cut detection)
   - Recommend transition types (crossfade, zoom, pan)
   - Calculate transition timing

3. FFmpeg Pipeline
   - Apply transitions between clips
   - Re-encode with consistent settings
   - Preserve quality (high bitrate, proper codec)

4. Audio Handling
   - Extract audio from clips (if present)
   - Crossfade audio at transitions
   - Sync to final video timeline

预期输出： 转场无缝衔接的精美视频

---

模型支持（2026）

图生视频模型

模型	质量	速度	显存	最适合	备注
LTX-2	★★★★★	中等	16GB+	制作级 4K 视频	原生 4K，音频+视频
Wan 2.2 MoE	★★★★★	慢	24GB+	电影级美学效果	首帧+末帧控制
Wan 2.1 14B	★★★★	慢	24GB	高质量	久经验证，稳定
Wan 2.1 1.3B	★★★	快	8GB	快速迭代	适合消费级设备
AnimateDiff V3	★★★	快	8GB	无限时长	动作 LoRA
SVD（Stable Video Diffusion）	★★★	中等	12GB	短片段	14-25 帧

转场效果

效果	使用场景	编码开销
交叉淡化	通用	低
运动模糊	高运动场景	中
放大/缩小	强调戏剧效果	中
向左/向右平移	建立场景	中
淡入黑场/从黑场淡出	章节分隔	低
自定义 LUT	调色	低

---

ComfyUI 实例管理

健康监控

# Auto-detected issues:
- Queue stalled (no progress for 5+ minutes)
- Memory leak (VRAM usage climbing)
- Process crashed (connection refused)
- API unresponsive (timeout on /queue endpoint)
- Disk full (output directory at capacity)

自动恢复操作

1. Soft Recovery (no restart)
   - Clear stuck queue items
   - Force garbage collection
   - Unload models from VRAM

2. Hard Recovery (restart required)
   - Save current queue state
   - Kill ComfyUI process gracefully
   - Wait for port release
   - Restart with same config
   - Restore queue from saved state

3. Emergency Fallback
   - Switch to backup ComfyUI instance
   - Redirect queue to instance on different port
   - Continue processing without data loss

多实例支持

# Run multiple ComfyUI instances for parallel processing
Instance 1: localhost:8188 (primary - I2V generation)
Instance 2: localhost:8189 (secondary - upscaling/post-processing)
Instance 3: localhost:8190 (backup - standby for failover)

# Load balancing strategy:
- Round-robin for equal workloads
- Priority-based for mixed tasks
- Failover for crashed instances

---

验证套件

生成前验证

✓ Check ComfyUI is running and responsive
✓ Verify models are loaded (UNET, VAE, CLIP)
✓ Confirm output directory has sufficient space
✓ Validate source images exist and are readable
✓ Check prompts are non-empty and formatted correctly
✓ Verify workflow JSON is valid

生成后验证

✓ Video file exists and is non-zero size
✓ Resolution matches expected (e.g., 768x1024)
✓ FPS matches expected (e.g., 16 or 25)
✓ Duration matches expected (e.g., 3-5 seconds)
✓ Codec is compatible (h264, h265)
✓ No corruption (can read all frames)
✓ Face consistency score >0.85 (if character video)
✓ Color histogram within expected range

质量指标

Metrics tracked:
- Face embedding distance (identity consistency)
- Optical flow magnitude (motion smoothness)
- Frame PSNR/SSIM (interpolation quality)
- Color histogram deviation (lighting consistency)
- Audio sync offset (if audio present)

---

错误处理与恢复

重试策略

1. Seed Randomization Retry
   - Failed generation? Try different seed
   - Max 3 attempts per keyframe
   - Track seeds that fail (avoid reuse)

2. Parameter Adjustment Retry
   - CFG too high causing artifacts? Lower it
   - Steps too low causing incompleteness? Increase
   - Resolution too high OOM? Downscale

3. Model Fallback Retry
   - Wan 2.2 14B OOM? Fall back to 1.3B
   - LTX-2 unavailable? Fall back to Wan 2.1
   - AnimateDiff motion broken? Switch motion LoRA

4. Checkpoint Resume
   - Save progress after each successful clip
   - Resume from last successful checkpoint
   - Skip already-generated clips

失败日志记录

logs/
├── 2026-02-16_pipeline.log       # Main pipeline log
├── 2026-02-16_comfyui.log        # ComfyUI stdout/stderr
├── 2026-02-16_validation.json    # Validation results
├── 2026-02-16_failures.json      # Failed attempts with reasons
└── 2026-02-16_recovery.json      # Recovery actions taken

---

目录结构

有序输出

project_name/
├── 00_keyframes/                 # Source keyframe images
│   ├── kf01_scene_description.png
│   ├── kf02_scene_description.png
│   └── ...
├── 01_clips/                     # Individual animated clips
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 02_validated/                 # Clips that passed validation
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 03_transitions/               # Intermediate files for transitions
│   ├── transition_001_002.mp4
│   └── ...
├── 04_final/                     # Final combined video
│   ├── final_video_v1.mp4
│   ├── final_video_v2.mp4        # After revisions
│   └── ...
├── logs/                         # Execution logs
├── metadata/                     # JSON metadata for each asset
└── manifest.json                 # Complete project manifest

---

工作流示例

示例 1：30 秒叙事视频（5 个关键帧）

# Configuration
project_name = "sage_character_reveal"
keyframes = 5
i2v_model = "wan_2.2_moe"
target_duration = 30  # seconds
fps = 16

# Pipeline execution
1. Generate 5 keyframes (IP-Adapter + LoRA)
   → sage_kf01_over_shoulder.png
   → sage_kf02_turning.png
   → sage_kf03_cardigan_fallen.png
   → sage_kf04_removing_bra.png
   → sage_kf05_topless.png

2. Validate keyframes
   → Face consistency: 0.92 ✓
   → Lighting consistency: 0.88 ✓
   → Pose progression: logical ✓

3. Queue I2V for each keyframe
   → clip_001: 6s @ 16fps (96 frames) ✓
   → clip_002: 6s @ 16fps (96 frames) ✓
   → clip_003: 6s @ 16fps (96 frames) ✓
   → clip_004: 6s @ 16fps (96 frames) ✓
   → clip_005: 6s @ 16fps (96 frames) ✓

4. Apply 0.5s crossfade transitions
   → Total: 30s - 2s (4 transitions × 0.5s) = 28s net

5. Export final video
   → sage_character_reveal_final.mp4 (30s, 768x1024, 16fps)

示例 2：批量处理 20 张图像

# Configuration
input_dir = "E:/ComfyUI/input/character_expressions"
i2v_model = "ltx_2"
motion_prompt = "gentle breathing, subtle movement, natural"
batch_size = 4  # Process 4 at a time

# Pipeline execution
1. Scan input directory
   → Found 20 PNG files

2. Queue 4 at a time to ComfyUI
   → Batch 1: expr_001.png → expr_004.png ✓
   → Batch 2: expr_005.png → expr_008.png ✓
   → Batch 3: expr_009.png → expr_012.png ✓
   → Batch 4: expr_013.png → expr_016.png ✓
   → Batch 5: expr_017.png → expr_020.png ✓

3. Validate each output
   → 19/20 passed (expr_011 failed - wrong resolution)
   → Retry expr_011 with corrected settings ✓

4. Export batch
   → 20 validated clips in output/expressions/
   → Generated contact sheet: expressions_preview.png

---

参考文件

详细指南

• references/keyframe-generation.md - 使用 IP-Adapter、LoRA 创建关键帧及保持一致性的技巧
• references/i2v-workflows.md - Wan 2.2、LTX-2、AnimateDiff、SVD 工作流模板
• references/concatenation.md - FFmpeg 命令、转场效果、音频处理
• references/validation.md - 质量指标、验证阈值、故障排查
• references/instance-management.md - ComfyUI 健康检查、重启脚本、多实例配置
• references/api-reference.md - ComfyUI API 端点、队列管理、工作流提交
• references/troubleshooting.md - 常见问题及解决方案

---

与其他技能集成

搭配使用：

• comfyui-character-gen - 生成初始关键帧并保持角色身份一致
• video-assembly - 进行高级剪辑和后期制作
• youtube-uploader - 制作完成后直接上传到 YouTube

---

高级功能

自适应质量

# Automatically adjust settings based on available resources
if vram_available > 24:
    use_model = "wan_2.2_moe_14b"
    resolution = (832, 1216)
    batch_size = 1
elif vram_available > 12:
    use_model = "wan_2.1_1.3b"
    resolution = (768, 1024)
    batch_size = 2
else:
    use_model = "animatediff_v3"
    resolution = (512, 768)
    batch_size = 4

进度跟踪

# Real-time progress updates
[Pipeline] Keyframe generation: 3/5 complete (60%)
[Pipeline] ETA: 12 minutes remaining
[I2V] clip_003 generating: 47/96 frames (49%)
[I2V] Current speed: 0.42 it/s
[Validation] clip_001: PASS ✓
[Validation] clip_002: PASS ✓

回滚与版本控制

# Automatically version outputs
output/
├── final_video_v1.mp4        # Initial render
├── final_video_v2.mp4        # After fixing clip_003
├── final_video_v3.mp4        # After adding transitions
└── final_video_final.mp4     # Approved version

# Rollback to previous version
rollback_to_version(2)  # Restore v2 as current

---

工作流生成

当有人要求创建视频制作工作流时：

1. 评估需求

   • 镜头／关键帧数量
   • 每个镜头的目标时长
   • I2V 模型偏好
   • 转场风格
   • 质量与速度之间的权衡

2. 生成流水线配置

   • 根据 VRAM／质量需求选择模型
   • 设置分辨率和 FPS
   • 设置验证阈值
   • 设置重试策略

3. 提供执行脚本

   • 用于提交 API 请求的 Python 脚本
   • 用于拼接视频的 FFmpeg 命令
   • 验证检查
   • 恢复流程

4. 监控并调整

   • 实时跟踪进度
   • 尽早检测故障
   • 应用恢复策略
   • 报告最终指标

---

最佳实践

对于关键帧视频

• 所有关键帧使用相同的 seed（保持一致性）
• IP-Adapter 权重设为 0.75-0.85（效果强但不僵硬）
• 在进行 I2V 之前验证关键帧（节省算力）
• 每个片段保持 4-8 秒（最佳时长）
• 使用 0.5-1 秒的交叉淡化转场（流畅但不拖沓）

对于批处理

• 分小批次处理（每次 4-8 个）
• 每批处理后立即验证
• 每个成功批次后保存检查点
• 为重要片段使用优先级队列

对于实例管理

• 每 30 秒监控一次队列深度
• 如果 5 分钟没有进展，则重启
• 在不同端口准备一个备用实例
• 记录所有重启事件以便调试

---

性能优化

RTX 50 系列（2026 年）

# ComfyUI launch flags for optimal performance
--highvram \
--fp8_e4m3fn-unet \
--reserve-vram 7 \
--use-pytorch-cross-attention

# Expected performance:
- Wan 2.2 14B: ~2-3 min per 5s clip (832x1216)
- LTX-2 4K: ~4-5 min per 5s clip (1920x1080)
- Wan 2.1 1.3B: ~1-2 min per 5s clip (768x1024)

AMD GPU（ROCm）

# ComfyUI v0.8.1+ has native ROCm support
# No special flags needed, just install ROCm drivers

---

技能演进

此技能会适配新的 I2V 模型和技术。发布新模型时：

1. 将模型规格添加到 references/i2v-workflows.md
2. 为新模型创建工作流模板
3. 更新主流水线中的模型选择逻辑
4. 使用示例项目进行测试
5. 记录性能特征

请参阅 references/evolution.md 了解更新流程。