claude-video-audio

claude-video-audio — Audio Processing

Pre-Flight

1. Run bash scripts/preflight.sh "$INPUT" "$OUTPUT"
2. Analyze audio streams: ffprobe -v error -select_streams a -show_entries stream=codec_name,channels,sample_rate,bit_rate -of json "$INPUT"

Loudness Normalization (EBU R128)

Two-pass workflow for accurate normalization:

# Pass 1: Measure current loudness
ffmpeg -i "$INPUT" -af loudnorm=I=-14:TP=-1.5:LRA=11:print_format=json -f null - 2>&1 | tail -12

Extract measured_I, measured_TP, measured_LRA, measured_thresh, offset from the JSON output.

# Pass 2: Apply normalization with measured values
ffmpeg -n -i "$INPUT" -af "loudnorm=I=-14:TP=-1.5:LRA=11:\
measured_I=MEASURED_I:measured_TP=MEASURED_TP:measured_LRA=MEASURED_LRA:\
measured_thresh=MEASURED_THRESH:offset=MEASURED_OFFSET:linear=true" \
  -c:v copy "$OUTPUT"

Target Loudness by Platform

Platform	Target LUFS	True Peak	Notes
YouTube	-14 LUFS	-1.0 dBTP	YouTube normalizes down, not up
Spotify	-14 LUFS	-1.0 dBTP	"Loud" mode = -11 LUFS
Apple Music	-16 LUFS	-1.0 dBTP	Sound Check
Podcast	-16 LUFS	-1.5 dBTP	Conversational clarity
Broadcast (EBU R128)	-23 LUFS	-1.0 dBTP	European standard
TikTok/Reels	-14 LUFS	-1.0 dBTP	Match YouTube

Noise Reduction

FFT-based (good for consistent background noise):

ffmpeg -n -i "$INPUT" -af "afftdn=nf=-25:nt=w" -c:v copy "$OUTPUT"

nf = noise floor in dB (lower = more aggressive). nt=w = white noise model.

Non-local means (better quality, slower):

ffmpeg -n -i "$INPUT" -af "anlmdn=s=7:p=0.002:r=0.002" -c:v copy "$OUTPUT"

Highpass/lowpass to remove rumble and hiss:

ffmpeg -n -i "$INPUT" -af "highpass=f=80,lowpass=f=12000" -c:v copy "$OUTPUT"

Silence Detection and Removal

Detect silence:

ffmpeg -i "$INPUT" -af "silencedetect=noise=-30dB:d=0.5" -f null - 2>&1 | grep silence

Returns silence_start and silence_end timestamps.

Remove silence (using Auto-Editor — much easier):

auto-editor "$INPUT" --margin 0.3s -o "$OUTPUT"

Manual silence removal with FFmpeg:

ffmpeg -n -i "$INPUT" -af "silenceremove=start_periods=1:start_silence=0.5:start_threshold=-30dB:detection=rms" -c:v copy "$OUTPUT"

Audio Extraction

# Extract as WAV (lossless)
ffmpeg -n -i "$INPUT" -vn -c:a pcm_s16le audio.wav

# Extract as AAC
ffmpeg -n -i "$INPUT" -vn -c:a aac -b:a 256k audio.m4a

# Extract as MP3
ffmpeg -n -i "$INPUT" -vn -c:a libmp3lame -b:a 320k audio.mp3

# Extract as Opus (most efficient)
ffmpeg -n -i "$INPUT" -vn -c:a libopus -b:a 128k audio.ogg

# Stream copy (fastest, keeps original codec)
ffmpeg -n -i "$INPUT" -vn -c:a copy audio.m4a

Audio Replacement

# Replace audio track entirely
ffmpeg -n -i video.mp4 -i new_audio.wav -c:v copy -c:a aac -b:a 192k \
  -map 0:v:0 -map 1:a:0 "$OUTPUT"

# Mix original audio with new audio (e.g., background music)
ffmpeg -n -i video.mp4 -i music.mp3 -filter_complex \
  "[0:a]volume=1.0[orig];[1:a]volume=0.3[music];[orig][music]amix=inputs=2:duration=first" \
  -c:v copy "$OUTPUT"

Audio Effects

Volume adjustment:

ffmpeg -n -i "$INPUT" -af "volume=1.5" -c:v copy "$OUTPUT"   # 1.5x louder
ffmpeg -n -i "$INPUT" -af "volume=-6dB" -c:v copy "$OUTPUT"   # 6dB quieter

Equalization:

# Boost bass, cut muddy mids, add presence
ffmpeg -n -i "$INPUT" -af "equalizer=f=80:t=q:w=1:g=3,equalizer=f=300:t=q:w=2:g=-2,equalizer=f=4000:t=q:w=1:g=2" -c:v copy "$OUTPUT"

Compression (reduce dynamic range):

ffmpeg -n -i "$INPUT" -af "acompressor=threshold=-20dB:ratio=4:attack=5:release=50" -c:v copy "$OUTPUT"

Limiter (prevent clipping):

ffmpeg -n -i "$INPUT" -af "alimiter=limit=0.95:level=false" -c:v copy "$OUTPUT"

Fade in/out:

ffmpeg -n -i "$INPUT" -af "afade=t=in:st=0:d=2,afade=t=out:st=OFFSET:d=2" -c:v copy "$OUTPUT"

Bass boost: -af "bass=g=5:f=100:w=0.5" Treble boost: -af "treble=g=3:f=4000:w=0.5"

Audio Format Conversion

Codec	FFmpeg Encoder	Recommended Bitrate	Use Case
AAC	aac	128-256k	Video, streaming
MP3	libmp3lame	192-320k	Universal playback
Opus	libopus	96-160k	Best efficiency, WebM
FLAC	flac	Lossless	Archival, editing
WAV	pcm_s16le	Uncompressed	Editing, processing
AC3	ac3	384-640k	Surround sound

Channel Layout

# Stereo to mono
ffmpeg -n -i "$INPUT" -ac 1 -c:v copy "$OUTPUT"

# Mono to stereo (duplicate)
ffmpeg -n -i "$INPUT" -ac 2 -c:v copy "$OUTPUT"

# Extract left channel only
ffmpeg -n -i "$INPUT" -af "pan=mono|c0=FL" -c:v copy "$OUTPUT"

Audio Sync Fix

# Delay audio by 0.5 seconds
ffmpeg -n -i "$INPUT" -itsoffset 0.5 -i "$INPUT" -map 0:v -map 1:a -c copy "$OUTPUT"

# Advance audio by 0.5 seconds
ffmpeg -n -i "$INPUT" -itsoffset -0.5 -i "$INPUT" -map 0:v -map 1:a -c copy "$OUTPUT"

Audio Ducking (Sidechain Compression)

Automatically lower music volume when speech is present.

Basic ducking (voice track ducks music track):

ffmpeg -n -i video_with_voice.mp4 -i music.mp3 -filter_complex \
  "[0:a]aformat=fltp:44100:stereo[voice];\
   [1:a]aformat=fltp:44100:stereo[music];\
   [music][voice]sidechaincompress=threshold=0.015:ratio=6:attack=200:release=1000:level_sc=1[ducked];\
   [voice][ducked]amix=inputs=2:duration=first:weights=1 0.4[out]" \
  -map 0:v -map "[out]" -c:v copy "$OUTPUT"

Parameters guide:

Parameter	Default	Purpose
threshold	0.015	Voice level that triggers ducking (lower = more sensitive)
ratio	6	How much to reduce music (higher = more reduction)
attack	200ms	How quickly music ducks down
release	1000ms	How quickly music returns after speech stops
weights	1 0.4	Voice volume : music volume in final mix

Ducking with existing audio streams (video has voice + separate music file):

ffmpeg -n -i "$INPUT" -i music.mp3 -filter_complex \
  "[1:a][0:a]sidechaincompress=threshold=0.02:ratio=8:attack=100:release=800[ducked];\
   [0:a][ducked]amix=inputs=2:duration=first:weights=1 0.3[out]" \
  -map 0:v -map "[out]" -c:v copy "$OUTPUT"

Tips:

• Lower threshold (e.g., 0.01) for quieter speech
• Higher ratio (e.g., 10-20) for more aggressive ducking
• Shorter attack (e.g., 50ms) for spoken word podcasts
• Longer release (e.g., 2000ms) for smoother music return
• Adjust weights to control the overall voice-to-music balance

Reference

Load references/audio.md for extended EBU R128 loudness details, advanced filter chains, and format-specific audio settings.