mcpField
M
modelslab-video-generation

modelslab-video-generation

github.com/modelslab创意与媒体1394 次安装36 周前更新
代码仓库

关于 modelslab-video-generation

modelslab-video-generation 是通过 ModelsLab API 生成和转换视频的技能，适合需要视频创作能力的创作者和开发者。它涵盖文本或图像生成视频、视频转换、口型同步及动作控制。

主要功能

  1. 01文本生成视频
  2. 02将静态图像制作成视频
  3. 03视频到视频转换
  4. 04音频与视频口型同步
  5. 05使用参考视频控制动作
  6. 06模型发现与异步结果处理

适用场景

  1. 01根据文本提示制作短片
  2. 02为静态图像添加动态效果
  3. 03转换现有视频的风格或内容
  4. 04将音频与视频人物口型同步

安装

复制下面的命令，在终端中运行。

$npx skills add https://github.com/modelslab/skills --skill modelslab-video-generation

安装后在 Claude Code 中输入/modelslab-video-generation确认已加载。

详细介绍

ModelsLab 视频生成

使用先进的视频生成模型，根据文本描述生成 AI 视频、为静态图像添加动画，或转换现有视频。

何时使用此技能

  • 根据文本描述生成视频
  • 为静态图像添加动画
  • 转换现有视频（视频转视频）
  • 将音频与视频进行口型同步
  • 根据参考视频应用动作控制
  • 创建短视频内容
  • 制作视频营销素材

可用 API（v7）

Video Fusion 端点
  • 文本转视频：POST https://modelslab.com/api/v7/video-fusion/text-to-video
  • 图像转视频：POST https://modelslab.com/api/v7/video-fusion/image-to-video
  • 视频转视频：POST https://modelslab.com/api/v7/video-fusion/video-to-video
  • 口型同步：POST https://modelslab.com/api/v7/video-fusion/lip-sync
  • 动作控制：POST https://modelslab.com/api/v7/video-fusion/motion-control
  • 获取结果：POST https://modelslab.com/api/v7/video-fusion/fetch/{id}

注意：v6 端点（/api/v6/video/text2video 等）仍可使用，但 v7 是当前版本。

发现视频模型

# Search all video models
modelslab models search --feature video_fusion

# Search by name
modelslab models search --search "seedance"
modelslab models search --search "wan"
modelslab models search --search "veo"

# Get model details
modelslab models detail --id seedance-t2v

文本转视频

import requests
import time

def generate_video(prompt, api_key, model_id="seedance-t2v"):
    """Generate a video from a text prompt.

    Args:
        prompt: Text description of the video
        api_key: Your ModelsLab API key
        model_id: Video model to use
    """
    response = requests.post(
        "https://modelslab.com/api/v7/video-fusion/text-to-video",
        json={
            "key": api_key,
            "model_id": model_id,
            "prompt": prompt,
            "negative_prompt": "low quality, blurry, static, distorted"
        }
    )

    data = response.json()

    if data["status"] == "error":
        raise Exception(f"Error: {data['message']}")

    if data["status"] == "success":
        return data["output"][0]

    # Video generation is async - poll for results
    request_id = data["id"]
    print(f"Video processing... Request ID: {request_id}")
    print(f"Estimated time: {data.get('eta', 'unknown')} seconds")

    return poll_video_result(request_id, api_key)

def poll_video_result(request_id, api_key, timeout=600):
    """Poll for video generation results."""
    start_time = time.time()

    while time.time() - start_time < timeout:
        fetch = requests.post(
            f"https://modelslab.com/api/v7/video-fusion/fetch/{request_id}",
            json={"key": api_key}
        )
        result = fetch.json()

        if result["status"] == "success":
            return result["output"][0]
        elif result["status"] == "failed":
            raise Exception(result.get("message", "Generation failed"))

        print(f"Status: processing... ({int(time.time() - start_time)}s elapsed)")
        time.sleep(10)

    raise Exception("Timeout waiting for video generation")

# Usage
video_url = generate_video(
    "A spaceship flying through an asteroid field, cinematic, 4K",
    "your_api_key",
    model_id="seedance-t2v"
)
print(f"Video ready: {video_url}")

图像转视频（为图像添加动画）

def animate_image(image_url, prompt, api_key, model_id="seedance-i2v"):
    """Animate a static image based on a motion prompt.

    Args:
        image_url: URL of the image to animate
        prompt: Description of desired motion/animation
        model_id: Video model for image-to-video
    """
    response = requests.post(
        "https://modelslab.com/api/v7/video-fusion/image-to-video",
        json={
            "key": api_key,
            "model_id": model_id,
            "init_image": [image_url],  # v7 expects array
            "prompt": prompt,
            "negative_prompt": "static, still, low quality, blurry"
        }
    )

    data = response.json()

    if data["status"] == "success":
        return data["output"][0]
    elif data["status"] == "processing":
        return poll_video_result(data["id"], api_key)
    else:
        raise Exception(data.get("message", "Unknown error"))

# Animate a landscape
video = animate_image(
    "https://example.com/landscape.jpg",
    "The clouds moving slowly across the sky, birds flying in the distance",
    "your_api_key",
    model_id="seedance-i2v"
)
print(f"Animated video: {video}")

视频转视频

def transform_video(video_url, prompt, api_key, model_id="wan2.1"):
    """Transform an existing video with a new style or content.

    Args:
        video_url: URL of the source video
        prompt: Description of desired transformation
    """
    response = requests.post(
        "https://modelslab.com/api/v7/video-fusion/video-to-video",
        json={
            "key": api_key,
            "model_id": model_id,
            "init_video": [video_url],  # v7 expects array
            "prompt": prompt
        }
    )

    data = response.json()
    if data["status"] == "processing":
        return poll_video_result(data["id"], api_key)
    elif data["status"] == "success":
        return data["output"][0]

口型同步

def lip_sync(video_url, audio_url, api_key, model_id="lipsync-2"):
    """Sync lip movements to audio.

    Args:
        video_url: URL of the video with a face
        audio_url: URL of the audio to sync to
    """
    response = requests.post(
        "https://modelslab.com/api/v7/video-fusion/lip-sync",
        json={
            "key": api_key,
            "model_id": model_id,
            "init_video": video_url,
            "init_audio": audio_url
        }
    )

    data = response.json()
    if data["status"] == "processing":
        return poll_video_result(data["id"], api_key)
    elif data["status"] == "success":
        return data["output"][0]

热门视频模型 ID

文本转视频
  • seedance-t2v - Seedance 文本转视频（BytePlus）
  • seedance-1.0-pro-fast-t2v - Seedance Pro Fast
  • wan2.6-t2v - Wan 2.6 文本转视频（Alibaba）
  • wan2.1 - Wan 2.1（ModelsLab 自研）
  • veo2 - Google Veo 2
  • veo3 - Google Veo 3
  • sora-2 - OpenAI Sora 2
  • Hailuo-2.3-t2v - Hailuo 2.3（MiniMax）
  • kling-v2-5-turbo-t2v - Kling V2.5 Turbo
图像转视频
  • seedance-i2v - Seedance 图像转视频
  • seedance-1.0-pro-i2v - Seedance Pro
  • wan2.6-i2v - Wan 2.6 图像转视频
  • Hailuo-2.3-i2v - Hailuo 2.3
  • kling-v2-1-i2v - Kling V2.1
口型同步
  • lipsync-2 - Sync Labs Lipsync 2
动作控制
  • kling-motion-control - Kling Motion Control
  • omni-human - OmniHuman（BytePlus）

浏览所有模型：https://modelslab.com/models

关键参数

参数描述推荐值
model_id视频生成模型（必填）参见上方模型表格
prompt视频内容的文本描述具体描述动作和场景
negative_prompt需要避免的内容"静止、低质量、模糊"
init_imagei2v 的源图像（数组）["https://..."]
init_videov2v 的源视频（数组）["https://..."]
init_audio口型同步／视频所用音频URL 字符串
width / height视频尺寸（512-1024）512、768、1024
duration视频时长（秒）4-30
aspect_ratio宽高比"16:9"、"9:16"、"1:1"
webhook异步通知 URLURL 字符串
track_id自定义跟踪标识符任意字符串

最佳实践

1. 编写以动作为重点的提示词
Bad: "A cat"
Good: "A cat walking through a garden, looking around curiously, sunlight filtering through trees"

Include: Action, movement, camera motion, atmosphere
2. 设定合理的预期
  • 视频通常为 4-30 秒
  • 生成时间因模型而异，从 30 秒到几分钟不等
  • 最适合制作短片段，而非完整作品
3. 处理异步操作
# Video generation is ALWAYS async
# Always implement polling or use webhooks
if data["status"] == "processing":
    video = poll_video_result(data["id"], api_key)
4. 使用 Webhook
payload = {
    "key": api_key,
    "model_id": "seedance-t2v",
    "prompt": "...",
    "webhook": "https://yourserver.com/webhook/video",
    "track_id": "video_001"
}

错误处理

try:
    video = generate_video(prompt, api_key, model_id="seedance-t2v")
    print(f"Video generated: {video}")
except Exception as e:
    print(f"Video generation failed: {e}")

资源

相关技能

  • modelslab-model-discovery - 查找和筛选模型
  • modelslab-image-generation - 生成用于 img2video 的图像
  • modelslab-audio-generation - 生成用于口型同步的音频
  • modelslab-chat-generation - 与 LLM 模型聊天
  • modelslab-webhooks - 高效处理异步操作

评论

支持 Markdown