media-generation

varg sdk - 声明式 AI 视频编排

基于 JSX 的 AI 视频生成。以声明式方式描述场景，自动渲染视频。

这是什么？

varg sdk 是一个声明式视频编排框架。无需手动调用 API、拼接片段和管理异步工作流，只需用 JSX 描述所需内容，运行时便会处理：

• 并行生成图像／视频／音频
• 自动缓存（重新渲染时会复用已缓存的资源）
• 在底层使用 ffmpeg 合成
• 提供商抽象（fal、elevenlabs、replicate）

可以把它想成视频领域的 React——你声明结构，引擎负责确定执行方式。

术语

术语	含义
Render	根容器——设置尺寸（TikTok 使用 1080x1920，YouTube 使用 1920x1080）
Clip	时间线片段，具有时长，包含视觉／音频图层
Image	静态图像——根据提示词生成或从文件加载
Video	视频片段——文本转视频或图像转视频动画
Music	背景音频——根据提示词生成或从文件加载
Speech	文本转语音，可选择音色
Title/Subtitle	带有定位的文字叠加层
Captions	根据 Speech 元素自动生成的字幕
Grid	用于多图像／视频网格的布局辅助工具

核心概念

<Render width={1080} height={1920}>  {/* tiktok dimensions */}
  <Music prompt="upbeat electronic" duration={10} />
  
  <Clip duration={3}>
    <Image prompt="cyberpunk cityscape" />
    <Title position="bottom">Welcome</Title>
  </Clip>
  
  <Clip duration={5}>
    <Video prompt="camera flies through neon streets" />
  </Clip>
</Render>

无需导入——渲染运行时会自动导入所有组件（Render、Clip、Video、Image、Music、Speech、Title、Grid 等）和提供商（fal、elevenlabs）。只需编写 JSX 并导出默认内容即可。

Video 组件

Video 同时支持文本转视频和图像转视频：

// text-to-video - generate from scratch
<Video prompt="cat playing piano in a jazz club" />

// image-to-video - animate an image with motion description
<Video 
  prompt={{
    text: "eyes widen, head tilts forward, subtle smile forming",
    images: [<Image prompt="portrait of woman" />]
  }}
/>

角色一致性

先生成图像，再将其传递给 Video/Animate，以确保不同场景中的角色保持一致：

const hero = <Image prompt="warrior princess, crimson hair, emerald eyes" />;

<Clip><Video image={hero} prompt="she draws her sword" /></Clip>
<Clip><Video image={hero} prompt="she leaps into battle" /></Clip>

基于参考内容生成（保持真人的相貌特征）：

<Image 
  prompt={{ 
    text: "same person, different scene", 
    images: ["https://reference-photo.jpg"] 
  }}
  model={fal.imageModel("nano-banana-pro/edit")}
/>

提示词指南（重要）

提示词越详细，效果越好。 Wan 2.5 视频模型对丰富、结构化的提示词响应效果要好得多。模糊的提示词会产生普通、低质量的输出。

四维公式

每条视频提示词都应包含以下维度：

[Subject Description] + [Scene Description] + [Motion/Action] + [Cinematic Controls]

主体描述

详细描述主要焦点：

• 外观、服装、特征、姿势
• 材质、纹理、颜色
• 情绪状态、表情

不佳： "a woman" 较好： "a young woman with shoulder-length auburn hair, wearing a worn leather jacket over a white t-shirt, her green eyes reflecting determination, jaw set with quiet resolve"

场景描述

层层补充环境细节：

• 地点、背景、前景
• 光线类型（柔光、硬光、轮廓光、顶光）
• 时段（黄金时刻、白天、夜晚、月光）
• 氛围（暖色调、低饱和度、高对比度）

不佳： "in a city" 较好： "on a rain-slicked tokyo street at night, neon signs reflecting pink and blue on wet pavement, steam rising from a nearby food stall, warm yellow light spilling from a ramen shop window"

动作／运动描述

明确动作的幅度和方式：

• 幅度（小幅动作、大幅动作）
• 速度（缓慢、快速、爆发式）
• 效果（玻璃破碎、头发飘动、尘埃颗粒）

不佳： "walking" 较好： "walking slowly through the crowd, shoulders hunched against the rain, one hand raised to shield her eyes, water droplets catching the neon light as they fall from her fingertips"

电影化控制

使用电影语言，打造专业效果：

镜头运动：

• camera push-in / camera pull-out
• tracking shot / dolly shot
• pan left/right / tilt up/down
• static camera / fixed camera
• handheld / steadicam

镜头类型：

• extreme close-up / close-up / medium close-up
• medium shot / medium wide shot
• wide shot / extreme wide shot
• over-the-shoulder shot

构图：

• center composition / rule of thirds
• left-side composition / right-side composition
• symmetrical composition
• low angle / high angle / eye-level

灯光关键词：

• soft light / hard light / edge light
• rim light / backlight / side light
• golden hour / blue hour / moonlight
• practical light / mixed light

风格关键词

添加视觉风格，指导艺术方向：

• cinematic / film grain / anamorphic
• cyberpunk / noir / post-apocalyptic
• ghibli style / anime / photorealistic
• vintage / retro / futuristic
• tilt-shift photography / time-lapse

音频/对白（适用于有对白的视频）

wan 2.5 支持原生音频生成：

• 对白格式：Character says: "Your line here."
• 台词要简短（每段 5 秒的视频不超过 10 个单词），以获得最佳口型同步效果
• 指定表达语气："speaking quietly"、"calling out over wind"
• 如需静音：添加 "no dialogue" 或 "actors not speaking"

图生视频提示词

使用参考图时，重点描述动作和镜头：

• 图像已经定义了主体/场景/风格
• 描述动态效果："running forward, hair flowing behind"
• 添加镜头运动："slow push-in on face"
• 控制动作强度："subtle movement" 与 "dramatic action"

示例：详细提示词

不好的提示词（模糊）：

<Video prompt="a warrior fighting" />

好的提示词（详细）：

<Video prompt="medium close-up, side light, golden hour. a battle-hardened samurai with a weathered face and grey-streaked topknot, wearing damaged traditional armor with visible scratches and dents. he raises his katana in a fluid motion, blade catching the dying sunlight, muscles tensing as he shifts into fighting stance. dust particles float in the amber light, his breath visible in the cold air. slow camera push-in on his focused expression, eyes narrowed with lethal intent. cinematic, film grain, anamorphic lens flare." />

示例 1：电影化角色视频

在多个戏剧性场景中保持角色一致，并搭配史诗感音乐。

warrior-princess.tsx（完整文件，无需导入）：

// Warrior Princess - Cinematic Character Video
// run: bunx vargai@latest render warrior-princess.tsx --verbose

const hero = (
  <Image prompt="portrait, soft light, center composition. warrior princess with flowing crimson hair reaching past her shoulders, piercing emerald eyes reflecting inner fire, wearing battle-worn silver armor with intricate celtic knot engravings, a thin scar across her left cheek from an old battle, expression of quiet determination mixed with weariness. ghibli style, painterly brushstrokes, warm color palette with golden undertones" />
);

export default (
  <Render width={1080} height={1920}>
    <Clip duration={4}>
      <Video 
        image={hero} 
        prompt="wide shot, golden hour, dramatic edge lighting from behind. she stands at the edge of a windswept cliff overlooking a vast misty valley with ancient ruins below, her crimson hair and tattered cape billowing dramatically in the wind, one hand resting on sword hilt at her side. slow camera pull-out revealing the epic landscape, dust particles catching the amber sunlight. cinematic, ghibli style, film grain, anamorphic lens" 
      />
    </Clip>
    <Clip duration={3}>
      <Video 
        image={hero} 
        prompt="medium close-up, dramatic side lighting with hard shadows. she draws her sword in one fluid motion, blade singing as it leaves the scabbard, polished steel catching the dying sunlight with a bright gleam. her eyes narrow with fierce focus, jaw set with lethal intent. camera tracks the sword movement upward. cinematic, ghibli style, shallow depth of field" 
      />
    </Clip>
    <Clip duration={3}>
      <Video 
        image={hero} 
        prompt="extreme close-up on her face, rim light from behind creating a golden halo effect. a single tear rolls down her scarred cheek as she whispers a prayer, eyes closed in concentration, lips barely moving. subtle camera push-in, dust motes floating in the light. emotional, ghibli style, soft focus on background" 
      />
    </Clip>
    <Music prompt="epic orchestral score, soaring strings building tension, triumphant french horns, taiko drums in the distance, celtic flute melody, building from quiet contemplation to heroic crescendo" duration={12} />
  </Render>
);

示例 2：带口播的 TikTok 产品视频

带语音、字幕和背景音乐的动画网红角色。

skincare-promo.tsx（完整文件，无需导入）：

// Skincare Product TikTok - Talking Head with Captions
// run: bunx vargai@latest render skincare-promo.tsx --verbose

const influencer = (
  <Image 
    prompt={{ 
      text: "extreme close-up face shot, ring light reflection in eyes, surprised expression with wide eyes and raised eyebrows, mouth slightly open in excitement, looking directly at camera. young woman with short platinum blonde hair, minimal makeup, silver hoop earrings, wearing oversized vintage band t-shirt. clean white background, professional studio lighting, tiktok creator aesthetic",
      images: ["https://reference-photo-url.jpg"]  // optional: reference for likeness
    }}
    model={fal.imageModel("nano-banana-pro/edit")}
    aspectRatio="9:16"
  />
);

const speech = (
  <Speech voice="bella" model={elevenlabs.speechModel("turbo")}>
    Oh my god you guys, I literally cannot believe this actually works! I've been using this for like two weeks and my skin has never looked better. Link in bio, seriously go get it!
  </Speech>
);

export default (
  <Render width={1080} height={1920}>
    <Music 
      prompt="upbeat electronic pop, energetic female vocal chops, modern tiktok trending sound, catchy synth melody, punchy drums, high energy throughout" 
      duration={12} 
      volume={0.4}
    />
    
    <Clip duration={3}>
      <Video 
        prompt={{
          text: "eyes widen dramatically in genuine surprise, eyebrows shoot up, mouth opens into excited smile, subtle forward lean toward camera as if sharing a secret. natural blinking, authentic micro-expressions",
          images: [influencer]
        }}
        model={fal.videoModel("wan-2.5")}
      />
      {speech}
    </Clip>
    
    <Clip duration={4}>
      <Video 
        prompt="product shot: sleek skincare bottle rotating slowly on marble surface, soft key light from above, pink and gold gradient background, light rays catching the glass, water droplets on bottle suggesting freshness. smooth 360 rotation, professional product photography, beauty commercial aesthetic"
        model={fal.videoModel("kling-v2.5")}
      />
      <Title position="bottom" color="#ffffff">LINK IN BIO</Title>
    </Clip>
    
    <Clip duration={3}>
      <Video 
        prompt={{
          text: "enthusiastic nodding while speaking, hands come into frame gesturing excitedly, genuine smile reaching her eyes, occasional hair flip, pointing at camera on 'go get it'",
          images: [influencer]
        }}
        model={fal.videoModel("wan-2.5")}
      />
    </Clip>
    
    <Captions src={speech} style="tiktok" activeColor="#ff00ff" />
  </Render>
);

示例 3：带元素主题的多场景视频网格

展示不同元素的 4 格自然主题视频网格。

four-elements.tsx（完整文件，无需导入）：

// Four Elements - 2x2 Video Grid
// run: bunx vargai@latest render four-elements.tsx --verbose

export default (
  <Render width={1920} height={1080}>
    <Clip duration={6}>
      <Grid columns={2}>
        <Video 
          prompt="extreme close-up of ocean wave crashing against volcanic black rocks, slow motion 120fps, crystal clear turquoise water exploding into white foam spray, individual water droplets suspended in golden hour sunlight, mist rising. cinematic, national geographic style, shallow depth of field on water texture"
          model={fal.videoModel("wan-2.5")}
        />
        <Video 
          prompt="intimate close-up of flames dancing in stone fireplace, warm orange and yellow tongues of fire licking weathered logs, glowing embers pulsing beneath, sparks occasionally rising. cozy cabin atmosphere, soft crackling implied, shallow focus on flame tips, hygge aesthetic"
          model={fal.videoModel("wan-2.5")}
        />
        <Video 
          prompt="macro shot of rain droplets hitting window glass, each drop creating expanding ripples and trails, city lights blurred into colorful bokeh beyond the glass, melancholic blue hour lighting. asmr visual aesthetic, slow motion, focus pulling between drops"
          model={fal.videoModel("wan-2.5")}
        />
        <Video 
          prompt="timelapse of cumulus clouds drifting across deep blue sky, dramatic god rays breaking through cloud gaps, cloud shadows moving across distant mountain range. epic landscape, wide angle, peaceful meditative mood, nature documentary cinematography"
          model={fal.videoModel("wan-2.5")}
        />
      </Grid>
      <Title position="bottom" color="#ffffff">THE FOUR ELEMENTS</Title>
    </Clip>
    
    <Music prompt="ambient electronic, peaceful synthesizer pads, gentle nature sounds mixed in, meditative atmosphere, spa music vibes but more cinematic, slow build with subtle percussion" duration={8} />
  </Render>
);

渲染

先预览（推荐工作流）

使用 --preview 仅生成图像/缩略图，而不渲染完整视频。这样迭代起来更快、成本也更低：

bunx vargai@latest render video.tsx --preview --verbose

预览帧满意后，渲染完整视频：

bunx vargai@latest render video.tsx --verbose

打开结果

渲染完成后，使用系统默认播放器打开视频：

open output/video.mp4

设置

需要在 .env 中配置 API 密钥：

• FAL_KEY - 图像/视频生成（https://fal.ai/dashboard/keys）
• ELEVENLABS_API_KEY - 音乐/语音（https://elevenlabs.io/app/settings/api-keys）

可用模型

// image generation
fal.imageModel("flux-schnell")        // fast, good quality
fal.imageModel("flux-pro")            // highest quality
fal.imageModel("nano-banana-pro/edit") // reference-based (character consistency)

// video generation
fal.videoModel("wan-2.5")             // balanced speed/quality
fal.videoModel("kling-v2.5")          // highest quality, 10s max
fal.videoModel("sync-v2")             // lipsync

// audio
elevenlabs.speechModel("turbo")       // fast TTS
elevenlabs.musicModel()               // music generation

关键属性参考

组件	关键属性
<Render>	width、height、fps
<Clip>	duration、transition={{ name: "fade", duration: 0.5 }}
<Image>	prompt、src、model、aspectRatio、resize、zoom
<Video>	prompt、src、model、duration、cutFrom、cutTo
<Music>	prompt、src、model、duration、volume、loop
<Speech>	voice、model、volume、children=text
<Title>	position、color、children=text
<Captions>	src（Speech 元素）、style（"tiktok"/"karaoke"）
<Grid>	columns、rows、children=Image/Video 数组

关于 varg.ai

varg 是一个生成式 AI 视频平台。该 SDK 使用声明式 JSX，通过 AI 生成的图像、动画和音频来组合视频场景。主要功能：

• 自动缓存（重新生成时使用缓存资源）
• 并行生成（片段同时渲染）
• 通过图像参考保持角色一致性
• 不依赖特定提供商（底层使用 fal、replicate、elevenlabs）
• 基于 ffmpeg 合成，输出专业品质的视频