video-to-sfx
Generate sound effects matched to a video using Sonilo — footsteps, impacts, ambience, foley — optionally scripted to specific timed segments, returning either the audio or a new video with the SFX muxed in. Use when the user has footage that needs sound design. For SFX from a text description alone, use the text-to-sfx skill; for music, use video-to-music.
sonilo-ai · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add sonilo-ai/skills --skill video-to-sfx技能说明
name: video-to-sfx
description: Generate sound effects matched to a video using Sonilo — footsteps, impacts, ambience, foley — optionally scripted to specific timed segments, returning either the audio or a new video with the SFX muxed in. Use when the user has footage that needs sound design. For SFX from a text description alone, use the text-to-sfx skill; for music, use video-to-music.
license: MIT
compatibility: "Requires Sonilo through either transport — the MCP server connected, or the sonilo CLI installed and signed in — plus credentials: a sonilo login sign-in, the hosted OAuth plugin, or SONILO_API_KEY. See the setup-api-key skill."
allowed-tools: Bash, Read, Write, mcp__sonilo__*
Sonilo Video-to-SFX
Hand Sonilo a video and it generates sound effects matching what it sees —
footsteps, impacts, ambience, UI sounds, whatever the scene calls for — or pin
specific sounds to specific moments with segments. Generation runs as an async
task on the backend; the tools poll internally and hand back the saved file.
Setup: See the setup-api-key skill.
⚠️ Cost: every tool below makes an API call that may incur charges. Only call it when explicitly requested.
Transport: MCP or CLI
Pick one at the start of the session and stay on it. Do not mix the two inside a single job, and do not announce the choice.
- Sonilo MCP tools visible in this session (
video_to_sfxand friends) — use them. This is the preferred path: it needs no shell, and it is the only one that survives a very long generation. If a call fails to authenticate — rather than failing on its inputs — this transport is not usable in this session: go to 2 instead of retrying it. - No usable Sonilo MCP tools, but
sonilo accountexits 0 — use the CLI commands below. Same API, same account, same credential file. Probe withsonilo account, notsonilo whoami: whoami exits 0 even when signed out, so it cannot tell the two states apart. - Neither — stop and run the setup-api-key skill. Do not call
api.sonilo.comwith curl to work around it; both transports handle uploads, polling and retries that a bare request does not.
Quick Start
MCP tool call (recommended)
video_to_sfx(
video_path="~/Desktop/action-scene.mp4",
prompt="Footsteps on gravel, distant traffic, a door slam"
)
Python (pip install sonilo)
from sonilo import Sonilo
client = Sonilo() # reads SONILO_API_KEY
foley = client.video_to_sfx.generate(video="action-scene.mp4", prompt="Footsteps on gravel, distant traffic, a door slam")
foley.save("foley.wav")
# video_to_video_sfx: get the video back with the effects muxed in
video = client.video_to_video_sfx.generate(video="action-scene.mp4", segments=[{"start": 0, "end": 2, "prompt": "footsteps on gravel"}])
video.save("with_sfx.mp4")
JavaScript / TypeScript (npm install sonilo)
import { SoniloClient } from "sonilo";
const client = new SoniloClient(); // reads SONILO_API_KEY
const foley = await client.videoToSfx.generate({
video: "./action-scene.mp4",
prompt: "Footsteps on gravel, distant traffic, a door slam",
});
// video_to_video_sfx: get the video back with the effects muxed in
const video = await client.videoToVideoSfx.generate({
video: "./action-scene.mp4",
segments: [{ start: 0, end: 2, prompt: "footsteps on gravel" }],
});
CLI (npm install -g sonilo-cli or pip install sonilo-cli)
sonilo video-to-sfx --video action-scene.mp4 --output foley.wav
Always async under the hood — the CLI submits and polls for you. --format accepts wav|mp3|aac|flac.
# the muxed video, from the CLI
sonilo video-to-video-sfx --video clip.mp4 --prompt "footsteps, distant thunder" --output foley.mp4
cURL (raw REST API, no MCP host)
curl -X POST "https://api.sonilo.com/v1/video-to-sfx" \
-H "Authorization: Bearer $SONILO_API_KEY" \
-F "video=@action-scene.mp4" \
-F "prompt=Footsteps on gravel, distant traffic, a door slam"
# -> {"task_id": "..."} poll GET /v1/tasks/{task_id} until status is succeeded/failed
Every call is task-based: the endpoint returns {"task_id": ...} (HTTP 202), and the result is fetched from GET /v1/tasks/{task_id} once status is terminal. The MCP tools do this polling for you and return the saved path directly — you only see the task_id if the call times out (see task-recovery).
Tools
| Tool | Description |
|---|---|
video_to_sfx(video_path? | video_url?, prompt?, segments?, audio_format?, output_directory?) | Generate SFX matched to a video. Returns audio only (not the source video). |
video_to_video_sfx(video_path? | video_url?, prompt?, segments?, output_directory?) | Same, but returns a new .mp4 with the SFX muxed in. |
Parameters
| Parameter | Type | Default | Notes |
|---|---|---|---|
prompt | string | — | Optional overall description (max 2000 chars) — omit it to let Sonilo interpret the video on its own. |
video_path | string | — | .mp4/.mov/.webm/.m4v/.gif (gif must be animated) — a narrower set than the music tools. Max 180s (3 min), subject to the account's upload-size cap. |
video_url | string | — | HTTPS/HTTP URL to a video. Exactly one of video_path/video_url. |
segments | list[dict] | — | Script SFX to specific time ranges: [{"start": float, "end": float, "prompt": str}, ...]. See rules below. Max 30 segments. |
audio_format | string | aac (.m4a) | wav, mp3, aac, or flac. video_to_sfx only (video-to-video always outputs .mp4). |
output_directory | string | SONILO_MCP_BASE_PATH | Absolute, or relative to the base path. |
segments rules
Validated by the backend before any charge — an invalid list is rejected with a 422/400 and nothing is billed:
- First segment's
startmust be0. - Segments must be contiguous: each
endmust e