video-to-sound
Generate music AND sound effects for a video in a single balanced, single-charge call using Sonilo. Use instead of calling the music and sound-effects skills separately for the same video — the two layers are mixed and ducked against each other by the backend. Returns a mixed audio track, or a new video with it muxed in.
sonilo-ai · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add sonilo-ai/skills --skill video-to-sound技能说明
name: video-to-sound
description: Generate music AND sound effects for a video in a single balanced, single-charge call using Sonilo. Use instead of calling the music and sound-effects skills separately for the same video — the two layers are mixed and ducked against each other by the backend. Returns a mixed audio track, or a new video with it muxed in.
license: MIT
compatibility: "Requires Sonilo through either transport — the MCP server connected, or the sonilo CLI installed and signed in — plus credentials: a sonilo login sign-in, the hosted OAuth plugin, or SONILO_API_KEY. See the setup-api-key skill."
allowed-tools: Bash, Read, Write, mcp__sonilo__*
Sonilo Video-to-Sound (Music + SFX Combined)
Generate a music bed and sound effects for a video in one call, balanced against each other and mixed by the backend — one charge instead of two separate generations. Use this whenever a video needs a full soundtrack (score + SFX), not just one or the other.
Setup: See the setup-api-key skill.
⚠️ Cost: makes one API call that may incur charges (billed once, not twice, even though it produces both layers). Only call when explicitly requested.
Transport: MCP or CLI
Pick one at the start of the session and stay on it. Do not mix the two inside a single job, and do not announce the choice.
- Sonilo MCP tools visible in this session (
video_to_soundand friends) — use them. This is the preferred path: it needs no shell, and it is the only one that survives a very long generation. If a call fails to authenticate — rather than failing on its inputs — this transport is not usable in this session: go to 2 instead of retrying it. - No usable Sonilo MCP tools, but
sonilo accountexits 0 — use the CLI commands below. Same API, same account, same credential file. Probe withsonilo account, notsonilo whoami: whoami exits 0 even when signed out, so it cannot tell the two states apart. - Neither — stop and run the setup-api-key skill. Do not call
api.sonilo.comwith curl to work around it; both transports handle uploads, polling and retries that a bare request does not.
Quick Start
MCP tool call (recommended)
video_to_sound(
video_path="~/Desktop/trailer.mp4",
music_prompt="Cinematic, building tension",
sfx_prompt="Footsteps, wind, distant thunder"
)
video_to_video_sound(
video_path="~/Desktop/trailer.mp4",
music_prompt="Cinematic, building tension"
)
Python (pip install sonilo)
from sonilo import Sonilo
client = Sonilo() # reads SONILO_API_KEY
mix = client.video_to_sound.generate(
video="trailer.mp4",
music_prompt="Cinematic, building tension",
sfx_prompt="Footsteps, wind, distant thunder",
)
mix.save("soundtrack.wav")
video = client.video_to_video_sound.generate(video="trailer.mp4", music_prompt="Cinematic, building tension")
video.save("scored.mp4")
JavaScript / TypeScript (npm install sonilo)
import { SoniloClient, download } from "sonilo";
import { writeFile } from "node:fs/promises";
const client = new SoniloClient(); // reads SONILO_API_KEY
const mix = await client.videoToSound.generate({
video: "./trailer.mp4",
musicPrompt: "Cinematic, building tension",
sfxPrompt: "Footsteps, wind, distant thunder",
});
await writeFile("soundtrack.wav", await download(mix.output_url));
const video = await client.videoToVideoSound.generate({
video: "./trailer.mp4",
musicPrompt: "Cinematic, building tension",
});
await writeFile("scored.mp4", await download(video.output_url));
CLI (npm install -g sonilo-cli or pip install sonilo-cli)
sonilo video-to-sound --video trailer.mp4 \
--music-prompt "Cinematic, building tension" --sfx-prompt "Footsteps, wind, distant thunder" \
--output soundtrack.wav
sonilo video-to-video-sound --video trailer.mp4 --music-prompt "Cinematic, building tension"
Unlike the music/sound-effects skills, both tools here have CLI commands. --stem music/--stem sfx (repeatable) additionally saves the individual layers next to the combined output.
cURL (raw REST API, no MCP host)
curl -X POST "https://api.sonilo.com/v1/video-to-sound" \
-H "Authorization: Bearer $SONILO_API_KEY" \
-F "video=@trailer.mp4" \
-F "music_prompt=Cinematic, building tension" \
-F "sfx_prompt=Footsteps, wind, distant thunder"
# -> {"task_id": "..."} poll GET /v1/tasks/{task_id}
Both endpoints are task-based (202 + poll), same as the sound-effects tools — the MCP tool waits for you.
Tools
| Tool | Description |
|---|---|
video_to_sound(video_path? | video_url?, music_prompt?, sfx_prompt?, segments?, preserve_speech?, ducking?, output_format?, variants_num?, output_directory?) | Generate and mix music + SFX for a video, returns a single audio file. |
video_to_video_sound(video_path? | video_url?, music_prompt?, sfx_prompt?, segments?, keep_original_sound?, preserve_speech?, ducking?, variants_num?, output_directory?) | Same, but returns a new .mp4 with the mixed soundtrack muxed in. By default the source's own audio is dropped — see keep_original_sound. |
Parameters
| Parameter | Type | Default | Notes |
|---|---|---|---|
video_path | string | — | .mp4/.mov/.webm/.m4v/.gif (gif must be animated). Max 180s (3 min), subject to the account's upload-size cap. |
video_url | string | — | HTTPS/HTTP URL. Exactly one of video_path/video_url. |
music_prompt | string | — | Style hint for the music bed (max 2000 chars). Optional — omit to let Sonilo decide. |
sfx_prompt | string | — | Description of the SFX layered over the music (max 2000 chars). Optional. |
segments | list[dict] | — | Per-segment SFX descriptions — same schema and validation rules as in the video-to-sfx skill. Max 30 segments. |
preserve_speech | bool | false | Keep the |