video-localization

视频本地化

使用 each::sense 将视频本地化并配音为多种语言。这项技能支持音频翻译、语音克隆、口型同步和字幕生成，助力内容面向全球发行。

功能

• AI 配音：将视频音频翻译并配音为任意语言
• 语音克隆：在不同语言中保留说话者的身份特征
• 口型同步：使口型动作与翻译后的音频同步
• 字幕生成：自动生成源语言或目标语言字幕
• 多语言版本：为多个市场创建本地化版本
• 营销内容本地化：为不同地区的受众调整广告内容
• 在线学习内容翻译：翻译培训和教育内容
• YouTube 本地化：为国际受众准备视频

快速开始

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this English video into Spanish with lip sync"}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/my-video.mp4"]
  }'

支持的语言

语言	代码	配音	字幕	语音克隆
英语	en	是	是	是
西班牙语	es	是	是	是
法语	fr	是	是	是
德语	de	是	是	是
意大利语	it	是	是	是
葡萄牙语	pt	是	是	是
中文	zh	是	是	是
日语	ja	是	是	是
韩语	ko	是	是	是
印地语	hi	是	是	是
阿拉伯语	ar	是	是	是

用例示例

1. 英语配音为西班牙语

通过自然的语音合成实现基础配音。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate and dub this English video into Spanish. Use a natural-sounding voice that matches the original speaker tone and energy."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/english-video.mp4"]
  }'

2. 视频翻译与口型同步

通过口型动作同步实现完整本地化，呈现专业效果。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this video from English to French with full lip sync. The lip movements should match the French audio perfectly. Maintain the original background music and sound effects."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/presentation-video.mp4"]
  }'

3. 字幕生成

生成准确的源语言或翻译后语言字幕。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Generate subtitles for this video. First transcribe the English audio, then create Spanish subtitles. Output both SRT files and a video with burned-in Spanish subtitles."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/interview.mp4"]
  }'

4. 多语言视频版本

通过单个工作流为多个市场创建本地化版本。

# Start multi-language localization session
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "I need to localize this product video into Spanish, French, and German. Start with Spanish dubbing with lip sync."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001",
    "video_urls": ["https://example.com/product-demo.mp4"]
  }'

# Continue with French version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Now create the French version with the same quality settings."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001"
  }'

# Continue with German version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Finally, create the German version."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001"
  }'

5. 配音语音克隆

在配音版本中保留原说话者的声音特征。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this video from English to Japanese using voice cloning. Clone the original speaker voice so they sound like themselves speaking Japanese. Maintain their unique voice characteristics, tone, and speaking style."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/ceo-message.mp4"]
  }'

6. 营销视频本地化

结合文化因素，为不同地区的受众调整营销内容。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this marketing video for the Latin American market. Translate to Spanish with a neutral Latin American accent, not Spain Spanish. Apply lip sync and ensure the messaging resonates with the target audience. Keep the energy and excitement of the original."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/brand-ad.mp4"]
  }'

7. 培训视频翻译

翻译企业培训和教育内容。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this employee training video from English to Mandarin Chinese. Use clear, professional narration suitable for corporate training. Include both dubbed audio and Chinese subtitles for accessibility. Technical terms should be accurately translated."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/onboarding-training.mp4"]
  }'

8. YouTube 视频本地化

为国际 YouTube 受众准备内容。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this YouTube video for Portuguese-speaking audiences (Brazilian Portuguese). Dub the narration with lip sync, generate Portuguese subtitles in SRT format, and create a localized thumbnail with Portuguese text overlay."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/youtube-tutorial.mp4"]
  }'

9. 广告创意本地化

为不同地区的市场调整广告内容。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this 30-second ad for the German market. Translate and dub to German with lip sync. Ensure the call-to-action is culturally appropriate. Keep the same pacing and emotional impact as the original English version."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/product-ad-30s.mp4"]
  }'

10. 在线学习内容翻译

翻译教育课程和学习材料。

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this e-learning module from English to Korean. The instructor should be dubbed with voice cloning to maintain authenticity. Generate Korean subtitles and ensure all on-screen text references are noted for manual localization. Educational terminology must be accurately translated."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/course-module-1.mp4"]
  }'

最佳实践

视频质量

• 源视频质量：使用高质量的源视频（建议 1080p 或更高分辨率）
• 音频清晰：确保源音频清晰，背景噪声尽可能少
• 单人说话：单人说话的内容效果最佳；多人说话需要更多处理

配音提示

• 自然语速：AI 会调整语速以匹配口型动作
• 保留情绪：翻译中会保留原始情感基调
• 背景音频：保留原始音乐和音效

口型同步优化

• 面部可见：确保说话者的面部清晰可见
• 光线均匀：光线充足的面部能带来更好的口型同步效果
• 正面拍摄：正面或略带角度的镜头效果最佳

字幕最佳实践

• 阅读速度：字幕显示时长适宜，便于阅读
• 换行：合理分段，提高可读性
• 时间同步：与音频和场景切换准确同步

模式选择

根据需求选择合适的模式：

模式	最适合	速度	质量
max	最终制作、交付客户、广播	较慢	最高质量的口型同步和语音
eco	草稿、内部审阅、快速预览	较快	质量良好，交付更快

建议：初步审阅时使用 eco 模式，最终交付时使用 max 模式。

多轮本地化工作流

使用 session_id 迭代本地化项目：

# Initial localization request
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this video into Spanish with lip sync"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz",
    "video_urls": ["https://example.com/source-video.mp4"]
  }'

# Request adjustments based on review
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "The voice sounds too formal. Can you make it more conversational and energetic?"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz"
  }'

# Add subtitles to the approved version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Add Spanish subtitles to this dubbed version"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz"
  }'

批量本地化

高效处理多个视频或语言：

# Use eco mode for batch processing drafts
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Create Spanish dubbed versions of these 3 marketing videos with subtitles"}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "eco",
    "video_urls": [
      "https://example.com/video1.mp4",
      "https://example.com/video2.mp4",
      "https://example.com/video3.mp4"
    ]
  }'

错误处理

错误	原因	解决方案
Failed to create prediction: HTTP 422	余额不足	在 eachlabs.ai 充值
Video too long	源视频超出限制	将视频拆分为较短的片段
Audio not detected	视频中没有语音	确保视频包含语音音频
Face not detected	没有可用于唇形同步的可见人脸	使用有可见说话者的视频
超时	生成过程复杂	将客户端超时时间设为至少 10 分钟

技术说明

• 客户端超时：视频处理时，将 HTTP 客户端超时时间设为至少 10 分钟
• 视频长度：为获得最佳效果，每个片段的最佳长度为 10 分钟以内
• 输出格式：生成的视频为采用 H.264 编码的 MP4 文件
• 字幕格式：字幕输出会提供 SRT 文件

相关技能

• each-sense - 核心 API 文档
• video-generation - 创建原创视频内容
• video-edit - 编辑和增强视频
• voice-audio - 语音合成和音频生成