FFmpeg workers: the parts nobody documents

The first time I ran an FFmpeg worker in production, it crashed on a single corrupted video and the retry loop tried to process the same file 47 times in 12 minutes. Here's what I learned, written down so the next person doesn't have to learn it the same way.

1. Sample thumbnails, never pick the first frame

The first frame of a video is almost never the best thumbnail. It's usually a black intro, a fade-in, or a logo. Sample 5–10 frames evenly distributed across the duration, pick the one with the highest visual variance, and ship that.

2. Classify errors before you retry

Not all FFmpeg failures are equal. The four you actually care about:

• Corrupted input — Invalid data found when processing input. Don't retry.
• Unsupported format — Format ... not detected. Don't retry.
• Missing source — file isn't there. Mark the job as failed, do not retry.
• FFmpeg crashed — anything else. Retry with backoff.

Retrying the first three will fill your queue, your disk, and your alerts.

3. Pin the FFmpeg version

apt install ffmpeg is a moving target. The behavior of the same command across two patch versions of FFmpeg can differ in surprising ways. Pin a version, bake it into your worker image, and rebuild the image when you intentionally upgrade.

4. Run it as a worker, not a server

A synchronous FFmpeg call inside a request handler is a request handler that hangs on large files. Run it as a worker fed by a queue or a status='pending' database table. The worker writes back via the same table. The request handler only enqueues and polls.

The full pattern lives in video-watermark-worker.