Back to Blog

Agent workflows

Give Your Video Agent a Timeline Contract Before Changing Frame Rate

Separate source timestamps, composition frames and delivery frame rate. Preserve cue timing when an agent changes export settings instead of treating fps as a cosmetic label.

OpenAgentSkillPublished:

An agent has built a thirty-second product demo. Someone asks for a different frame rate. The agent changes one setting and exports again. Is it still a thirty-second demo, and does the final callout still arrive at the intended moment?

The answer depends on what changed: a frame-based composition, the sampling of an existing video, or the interpretation of source timing. Treating all three as an fps label makes review unnecessarily confusing.

This guide is for creators and developers who combine recorded clips with programmatic titles, callouts and narration. It proposes a timing contract before tool selection or export.

Methodology: separate three clocks

We reviewed Remotion's composition configuration reference and FFmpeg's filter and inspection documentation. No clip was rendered or played for this article. The numerical example below is arithmetic, and the review exercise is a proposed test.

Keep three concepts separate: the source media's presentation timing, the composition's frame grid, and the final delivery's timing. A source can contain its own timestamps while the surrounding composition places events on a fixed grid.

Remotion exposes composition fps and durationInFrames through useVideoConfig. Inside a Sequence, durationInFrames can instead describe that sequence. This is a useful reminder to identify which timeline a number belongs to. Remotion useVideoConfig.

Do not infer the complete source-media timing behavior from a composition setting alone.

A 900-frame example reveals the ambiguity

Consider a fictional composition with 900 frames at 30 fps. Its nominal duration is 900 divided by 30: thirty seconds. Keeping 900 frames while changing the composition to 60 fps describes fifteen seconds, not a smoother thirty-second version.

If the intention is to preserve thirty seconds at 60 fps, the composition needs 1,800 frames. But updating its total duration is not enough if the callouts and scene boundaries remain hard-coded to their old frame positions.

A callout starting at frame 300 originally begins ten seconds into the composition. On the new 60 fps grid, frame 300 corresponds to five seconds. The correct new frame position for the same ten-second cue would be 600.

These calculations do not predict what every editor or generator will do automatically. They show why the agent needs to state whether event positions are stored in seconds, frames, or source-media coordinates before editing the configuration.

Conversion is not the same operation

FFmpeg's fps filter converts video to a specified constant frame rate by duplicating or dropping frames as needed. Its documented settings also include timestamp rounding and start-time behavior. FFmpeg fps filter.

That conversion operates on an existing video's timing; it is not the same as shortening a frame-based composition by changing its fps while retaining the old frame count. Nor should duplicating frames be presented as recovering motion detail that was never recorded.

For our proposed demo workflow, ask the agent to explain the chosen operation in ordinary language: preserve the existing duration while producing a different frame cadence, or deliberately retime the scene. A request for one does not authorize the other.

Keep a non-destructive source copy and name the derived export separately. Do not use a generic frame-rate conversion as a promised cure for every audio-sync problem. The cause may lie elsewhere in the recording or edit.

Write a cue ledger before making the change

A compact ledger for the fictional demo might list an opening title at zero seconds, a feature reveal at ten seconds, and a closing callout at twenty-five seconds. Add the desired end time and the narration segment each visual cue accompanies.

For every cue, record whether it follows absolute composition time, a local scene time, or an identifiable event in the source recording. These relationships matter when a middle section is trimmed: an absolute cue and a source-linked cue may need different adjustments.

Choose a rounding policy when times do not land exactly on the output frame grid. Convert from the agreed timing source consistently rather than repeatedly rounding intermediate values across several edits.

This ledger is intentionally smaller than a full editing project. It gives the reviewer a few meaningful anchors that connect the user's request to the implementation.

Inspect the exported artifact, not just project settings

ffprobe can report stream information, show per-frame information and count frames. Those options support inspection of the actual file rather than relying exclusively on the renderer's configured values. ffprobe documentation.

Our recommendation is to capture the selected video and audio streams, available start-time and duration information, and the relevant frame timing around a suspicious cue. An average frame-rate summary is not, by itself, a description of every frame's presentation interval.

Compare the final cue positions to the ledger, then play the transitions and the ending with audio. A numeric report can identify an inconsistency, but it does not judge whether a click highlight lands convincingly on the action or whether a spoken phrase has been clipped.

Keep inspection bounded for long media. Document which sections were sampled and do not call a sample review a complete viewing.

Limitations and an acceptance exercise

Use a short non-sensitive fixture with recognizable beginning, middle and ending events. Ask the candidate workflow to produce a second frame-rate delivery while preserving the agreed duration and cue times. Then ask for an intentional shorter cut as a separate task.

Compare the two requests and their change records. The workflow should distinguish conversion from editing, identify any timing recalculation and report unresolved discrepancies. These are proposed acceptance criteria, not successful results from a particular skill.

Source timestamps, variable frame timing, codec behavior and destination playback can complicate the result. Keep exact rates where relevant instead of casually treating nearby values as identical. This article does not prescribe one export rate for every platform.

Our caption handoff guide covers the separate task of carrying reviewed text into the final release. Use the skills directory to shortlist production workflows, but evaluate them against the same timing request. A different fps value is not a complete specification of what the viewer should see and hear.

Give Your Video Agent a Timeline Contract Before Changing Frame Rate | OpenAgentSkill