ai-media
Use when a creative goal must become a finished media file: pick and order generative-media models per modality — AI voiceover, image-to-video clips, score — then glue them with ffmpeg (mux, duck, loudnorm, concat). NOT still-image generation/editing (that is `replicate-images`); NOT code-rendered React compositing (that is `remotion-video`).
ericrisco · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add ericrisco/rsc-harness --skill ai-media技能说明
name: ai-media
description: "Use when a creative goal must become a finished media file: pick and order generative-media models per modality — AI voiceover, image-to-video clips, score — then glue them with ffmpeg (mux, duck, loudnorm, concat). NOT still-image generation/editing (that is replicate-images); NOT code-rendered React compositing (that is remotion-video)."
tags: [ai-media, text-to-speech, image-to-video, voiceover, music-generation, ffmpeg, media-pipeline, elevenlabs]
recommends: [replicate-images, fal, replicate, remotion-video, video-shorts]
origin: risco
ai-media
You are the cross-modal director. You decide which generative-media model to call per modality, in what order, with what params, then assemble the pieces with ffmpeg into one finished file. You do not own a single provider's API surface and you do not prompt still images — you orchestrate and glue.
Pipeline shape — decide what the goal needs
Map the goal to modalities and an ordered step list, and lock that plan before you generate a single asset — media generation is slow and metered, so a re-roll of a 10 s Veo clip or a 90 s music track costs real money and minutes. Fixing the scene list, aspect ratio, target loudness and model per modality first is cheaper than discovering at mux time that your clips are 9:16 and your VO is the wrong sample rate. The "delegate to" column is where the actual call mechanics live — you pick the model and params, those skills run the call.
| Goal | Needs | Ordered steps | Delegate calls to |
|---|---|---|---|
| Narrated explainer | stills + img→video + VO + music | script → per-scene stills → clip per scene → VO → music → conform → concat → mix+duck → loudnorm → MP4 | replicate-images, fal/replicate |
| Product teaser (1 hero) | 1 still + img→video + music | still → clip → music → mix → loudnorm → MP4 | replicate-images, fal/replicate |
| Faceless short | stills + img→video + VO + music + captions | (explainer pipeline) + burn captions | replicate-images; ../video-shorts/SKILL.md for the script |
| Just a voiceover | VO only | script → TTS → loudnorm | — |
| Just a clip from a still | img→video only | still (input) → clip | fal/replicate |
| Code-rendered explainer | none of the above | render from React/TS | stop — route to remotion-video |
If the video is rendered from data/code (charts, timelines, JSON-driven scenes), this is not your job → ../remotion-video/SKILL.md. You handle model-generated + ffmpeg-glued.
Modality 1 — Voice (TTS)
ElevenLabs Python SDK. The call is convert(text, voice_id, model_id, output_format); auth via ELEVENLABS_API_KEY.
from elevenlabs.client import ElevenLabs
client = ElevenLabs() # reads ELEVENLABS_API_KEY
audio = client.text_to_speech.convert(
text="Your narration script here.",
voice_id="JBFqnCBsd6RMkjVDRZzb",
model_id="eleven_multilingual_v2", # final-quality VO
output_format="mp3_44100_128", # codec_samplerate_bitrate
)
with open("vo.mp3", "wb") as f:
for chunk in audio:
f.write(chunk)
Pick the model tier by what the job needs:
| Model | When | Latency | Cost lever |
|---|---|---|---|
eleven_v3 | most expressive, hero final VO — verify availability first (see caveat) | higher | most credits/char |
eleven_multilingual_v2 | high-quality multilingual VO (default for finals) | medium | medium |
eleven_flash_v2_5 | real-time / batch / scale / drafts | ~75 ms | cheapest |
Do not hardcode eleven_v3 blind. It shipped to the API in alpha (Aug 2025) and the docs model list now carries it, but the text_to_speech.convert API reference still documents the default as eleven_multilingual_v2 and does not enumerate eleven_v3 as a guaranteed value. Before you build a final pass on it, confirm it returns from GET /v1/models for your key (or just call once and check) — otherwise default to eleven_multilingual_v2, which is the safe, always-available hero tier.
output_format is codec_samplerate_bitrate — e.g. mp3_44100_128, mp3_22050_32. Match the VO sample rate to your assembly target, do not master the VO loud and hope.
Bad → Good:
- Bad: generate VO at
mp3_22050_32, then mux onto a 48 kHz video — ffmpeg silently resamples, you get artifacts and a level mismatch. - Good: generate VO at the rate you will master at (e.g.
mp3_44100_128), and set final loudness withloudnormin assembly, not by cranking the TTS.
TTS is billed per character/token (~0.5–1 credit/char on the Flash/Turbo lines). ElevenLabs cut TTS API pricing up to 55% on 2026-05-07 (e.g. Flash on Creator $0.11→$0.05 / 1k tokens) — that figure is TTS-specific, not the Music cut. Pricing staling fast: these are point-in-time numbers from elevenlabs.io/pricing/api as of 2026-06-02 — re-check the page before quoting a budget. Shorter scripts and Flash on drafts are the cost levers.
Modality 2 — Image-to-video
The still is an input, not your output. Generate or edit the source image in ../replicate-images/SKILL.md, then animate it here. Reality check: every serious 2026 model does 1080p or native 4K — resolution is no longer the differentiating axis. The hard limit is per-generation duration (~5–15 s, model-dependent). Long pieces are one clip per scene, then concat — never one long take.
Durations below are from each vendor's own pages (as of 2026-06; see references/models-and-params.md for the citations) — they move with releases, so verify on the catalog before a final run:
| Model | Duration | Aspect / max res | Control surface | Native audio | Open-source |
|---|---|---|---|---|---|
| Google Veo 3.1 | 8 s / generation | 16:9 / 9:16, up to 4K | high | yes — synced 48 kHz dialogue/SFX | no |
| Kling 3.0 | up to 15 s | flexible, 4K | st |