Video Tutorial

What Is AI Frame Interpolation?

AI frame interpolation generates new in-between video frames so motion plays back more smoothly. This tutorial explains how it works, when to use it, what its limits are, and how to run it on Media Manipulator.

1. What is frame interpolation?

Frame interpolation increases the frame rate of a video by inserting new images between the original frames. A 30fps clip has 30 frames per second; doubling its frame rate means producing 30 new frames per second to sit between the existing ones. There are several approaches:

• Frame duplication — copy the previous frame so motion looks the same but the playback rate matches the target.
• Frame blending — fade between two adjacent frames to soften the boundary. Cheap but mushy.
• Motion interpolation — estimate motion vectors and warp frames forward (e.g. FFmpeg’s minterpolate).
• AI interpolation — a neural network (e.g. RIFE) predicts the in-between frame from the two real frames around it. Usually the cleanest result for non-rigid motion.

Media Manipulator’s AI Frame Interpolation tool uses RIFE via rife-ncnn-vulkan on our own GPU server.

2. Why people use frame interpolation

• Smoother motion on modern displays and social platforms.
• Lifting 24/30fps footage to 60fps.
• Lifting 60fps footage to 120fps for high-refresh playback or editing.
• Cleaner motion previews for trailers, gameplay, sports, screencasts, and vlogs.
• Restoring older, low-FPS clips so they feel less stuttery.
• Producing smoother short cuts for sharing on social.

3. How AI frame interpolation works (high level)

Given two adjacent frames, the model estimates how every pixel moves between them — a process related to optical flow. It then uses that motion to construct one or more new frames placed at fractional time positions between the two originals. The neural network learns common motion patterns from training data, so it tends to handle smooth, natural motion well.

Hard cases — fast motion, occlusion (one object passing in front of another), thin structures like hair and wheels, and scene cuts — are inherently harder because the model has less reliable evidence about what should happen in between. Those regions are where artifacts tend to appear.

4. AI interpolation vs basic FPS conversion

Basic FPS conversion (whether through FFmpeg’s fps filter or a hardware-accelerated equivalent) usually just duplicates, drops, or blends frames to hit a target frame rate. The total number of frames per second matches the target, but new motion information is not added — the eye still sees only the original capture intervals.

AI interpolation creates net-new frames whose contents are estimated rather than copied. The trade-off:

• AI usually looks much smoother than duplication or blending.
• AI can warp or hallucinate details on hard regions.
• Basic FPS conversion is faster but rarely feels truly smoother.

5. When to pick 48, 60, or 120 FPS

• 48 FPS — a subtle 2× lift on 24fps cinematic footage. Less aggressive than 60fps and keeps a bit of the original cadence.
• 60 FPS — common smooth playback rate for the modern web. A good first target for 24/30fps source.
• 120 FPS — high-refresh displays, gameplay/sports content, or downstream slow-motion editing. Heavier processing cost and a larger file.

6. Quality and max-height tradeoffs

• Lower max processing height → faster turnaround and lower GPU memory pressure. Great for quick tests.
• Higher max processing height → more detail preserved, but slower and heavier.
• Low quality preset → faster encode, larger CRF, smaller file.
• High quality preset → slower encode, lower CRF, sharper output.

For a first run, try 720p max height with the medium quality preset.

7. Best practices

• Test on a short clip first — interpolation cost scales with duration and resolution.
• Avoid heavy scene-cut content. Cuts often produce a brief warp at the boundary.
• Preview the result before publishing — interpolation can look subtly different from a real high-FPS capture.
• Pick clear, steady source footage when possible.
• Keep the original — interpolation is irreversible re-encoding.
• Try 60fps before 120fps unless you specifically need 120fps.

8. Limitations and artifacts

• Warping around hands, hair, wheels, and other fast-moving thin structures.
• Ghosting around occlusions where one object passes in front of another.
• Brief glitches at scene cuts.
• Motion blur mismatch — synthesized frames don’t share the source camera’s shutter behavior.
• Existing compression artifacts (banding, blocking) can become more visible at higher frame rates.
• Long videos take a lot of GPU time; we cap duration to keep the free tool stable.
• AI interpolation does not recover real-world motion that was never captured — it estimates a plausible in-between frame.

9. How to use the Media Manipulator AI Frame Interpolation Tool

1. Open the AI Frame Interpolation tool.
2. Upload a video.
3. Scroll to the AI Video Tools panel and pick AI Frame Interpolation.
4. Choose a target FPS — 48, 60, or 120.
5. Pick a quality preset and a max processing height (start at 720p / medium).
6. Leave Preserve audio on unless you want a silent output.
7. Submit the conversion and wait for the job to finish.
8. Download the resulting MP4 and preview it.

FAQ

• Does frame interpolation improve video quality?
  It improves motion smoothness, not resolution or sharpness.
• Is AI frame interpolation the same as increasing FPS?
  Both target a higher frame rate, but AI interpolation synthesizes new frames instead of duplicating or blending.
• Can I convert 30fps to 60fps?
  Yes — it’s the most common target.
• Can I convert 60fps to 120fps?
  Yes, but the processing cost roughly doubles versus 60fps.
• Why can frame interpolation create artifacts?
  Motion estimation has to guess on hard regions (occlusions, fast motion, scene cuts).
• What output format does the tool create?
  MP4 (H.264 + AAC) in v1.
• Is 60fps always better than 30fps?
  Not for cinematic looks — pick what suits the content.
• Is AI frame interpolation better than FFmpeg FPS conversion?
  For most clips, yes — but minterpolate is faster and still useful as a quick fallback.
• Why do scene cuts sometimes look strange after interpolation?
  The model assumes adjacent frames belong to the same motion sequence.