meme-creator

Meme Creator

Turn a video clip into a meme: images (logos, avatars, stickers) glued onto moving objects so they follow the motion frame by frame, delivered as MP4 + GIF.

The failure mode that ruins this is not tracking drift — it is gluing the wrong person's avatar on screen because a name was bound to the first plausible account found. So step 0 is identity, not tooling.

For a static image meme, skip tracking entirely: put one manual_keys entry per target ([[1, [x, y, w, h]]]) in the overlays config and render just that frame — --tracks is optional when every position comes from keys.

Requirements

• ffmpeg on PATH
• uv — all bundled Python scripts carry inline dependencies; run them with uv run <script> and deps resolve themselves
• yt-dlp only when downloading (uv run --with yt-dlp yt-dlp ...)
• Chrome/Chromium only when rasterizing an SVG logo (asset-binding.md)
• Script paths below are written scripts/… relative to this skill's bundle directory. Resolve the bundle path once (SKILL_DIR=<path to this skill>) and run "$SKILL_DIR"/scripts/make_sheets.py … — your working directory is the scratch dir, not the bundle.

Pipeline

Work inside a scratch directory (/tmp/<meme-name>/ or similar), deliver only the final MP4 + GIF.

0. Bind identities before fetching assets

When the user names who/what goes on screen ("Codex, Claude, and Tibo's avatar"), every ambiguous name must survive the disambiguation gate in references/asset-binding.md — enumerate candidates with a handle-free search, discriminate from the request context (the peer entities listed alongside are the strongest signal), and ask the user when context cannot decide. That file also covers logo/avatar sourcing, SVG→PNG with real transparency via headless Chrome, and badge styling. Fetch and eyeball every image before compositing it.

1. Acquire the source

Local file: use it. URL: yt-dlp handles bilibili (incl. b23.tv short links), YouTube, and most sites:

uv run --with yt-dlp yt-dlp -f "bv*+ba/b" --merge-output-format mp4 -o "source.%(ext)s" "<url>"

If a domestic-CN site fails with a proxy tunnel error, retry with --proxy "" (a local HTTP proxy breaks some domestic CDNs); if an international site times out, retry with the local proxy.

Check duration: ffprobe -v error -show_entries format=duration -of csv=p=0 source.mp4

2. Locate the segment

Never track "the whole video." Find the exact scene with a 1–2 fps contact sheet, then expand around it:

ffmpeg -v error -ss <start_s> -t 40 -i source.mp4 -vf "fps=1,scale=240:-1,tile=5x8" -frames:v 1 contact.png

Read the sheet, pick the beat boundaries (entrance → action → exit), and cut the final segment ±1 s of padding. Extract the frame sequence at native fps:

mkdir -p seq && ffmpeg -v error -ss <seg_start> -i source.mp4 -t <seg_dur> seq/f_%04d.png
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 source.mp4

3. Prep overlay assets

One image per target, PNG with alpha. Unify mixed assets (app icon + logo mark

• photo avatar) as circle-white badges — see asset-binding.md for why and for the avatar circular-mask handling.

4. Track the targets

Full procedure and failure taxonomy: references/tracking-playbook.md. Short form:

1. Grid one frame where all targets are fully visible: uv run scripts/make_sheets.py grid --frames 'seq/f_%04d.png' --index 30 --out grid.png and read each target's [x, y, w, h] box off the grid.
2. Write the tracking config (targets + init frame), run: uv run scripts/track_boxes.py --frames 'seq/f_%04d.png' --config track.json --out tracks.json --last <N>
3. Verify: uv run scripts/make_sheets.py tile --frames 'seq/f_%04d.png' --tracks tracks.json --start 1 --end <N> --step-frames 15 --out verify.png and look at every tile.
4. Fix what the tiles show: re-anchor later segments (shot changes, scale blowups, drift), or take over a smooth long shot with manual_keys. Expect 2–3 rounds. Then mark visible/fade_in/fade_out for every exit, entrance, and empty-stretch — a logo parked on an empty frame is the most visible possible bug.

5. Render

uv run scripts/render_overlay.py --frames 'seq/f_%04d.png' --tracks tracks.json \
  --overlays overlays.json --out 'out/o_%04d.png' --last <N>

Spot-check a rendered contact sheet (make_sheets.py tile over out/) before encoding — much cheaper than re-encoding.

6. Encode + size budget

scripts/encode_outputs.sh --frames 'out/o_%04d.png' --fps <native_fps> \
  --source source.mp4 --ss <seg_start> --t <seg_dur> --mp4 meme.mp4 \
  --gif meme.gif --gif-width 400 --gif-fps 10 --gif-colors 96

GIF size is driven by frame count × dither noise, not palette size. To fit a ~10 MB chat-app budget, lower knobs in this order: --gif-fps (15→12→10), then --gif-width (480→400), then --gif-colors (128→96).

7. Verify the encoded deliverables, then deliver

Extract frames from the final MP4 and the final GIF (ffmpeg -ss <t> -frames:v 1) and look at them — the earlier checks verified intermediates; this one verifies what the recipient will actually see. Deliver both files by their exact paths.

Troubleshooting

Symptom	Cause	Fix
yt-dlp: Tunnel connection failed 503 on a domestic site	local HTTP proxy intercepting	add --proxy ""
cairosvg/svglib crash: no library called cairo	missing system cairo	rasterize SVG via headless Chrome (asset-binding.md)
Tracker dies exactly at a shot change	CSRT cannot cross cuts	new segments entry at the new shot's first frame
Box drifts slowly off the target over ~1 min	template pollution	re-anchor with the tracker's own last-good box, or manual_keys
Logo parked mid-frame after subject exits	missing visibility window	fade_out before the exit, no visible range after
GIF massively over budget	dither noise × frames	lower --gif-fps first, then width, then colors
Logo looks fine in PNGs but wrong in the GIF	checked intermediates only	always verify frames from the encoded file (step 7)
Encode fails: Failed to set value '1:a' for option 'map'	source clip has no audio stream	the bundled script maps 1:a? (optional); if running ffmpeg by hand, do the same or drop --source

References

• references/asset-binding.md — identity disambiguation gate, logo/avatar sourcing, transparency, badge styling
• references/tracking-playbook.md — CSRT failure taxonomy, re-anchor recovery, manual keyframes, fades, verify loop