Count, interval, keyframe, timestamp: what the four modes actually do

The modes are not three ways to do the same thing with different settings. They answer different questions, and choosing the wrong one produces a set of stills that is complete in a way you did not want.

The pattern is that count and interval distribute coverage, while timestamps concentrate it. If you know when the thing happened, timestamps is not merely the better choice — it is the only one that can be right. If you do not know, distributing coverage is the only honest option, and the question becomes how to distribute it so the event cannot hide between samples.

Count mode divides the duration by N + 1 and requests the N interior points. For a 60-second clip and N = 5, the divisor is 6, so the requested times are 10, 20, 30, 40 and 50 seconds.

Dividing by N + 1 rather than N − 1 is a small decision with a visible consequence: 0:00 and 0:59.9 are never requested. That is usually right. The first and last frames of an export are disproportionately likely to be a fade from black, a title card, or a frame the editor trimmed to the boundary, and including them makes every overview look like it starts with a black rectangle. But it is a decision, not a law, and it is the reason a count-mode overview of a clip whose important moment is in the opening second will show you nothing.

The other property worth knowing: the spacing is uniform in time, not in content. A five-frame overview of a lecture is five evenly spaced frames, four of which may be the same slide. Count mode cannot know that, and no amount of tuning N fixes it. If the clip has chapters, they are a better source of sample points than arithmetic.

Interval mode requests 0, s, 2s and so on while the running value is strictly less than the duration. Two consequences follow directly from that loop condition.

• The final partial segment is never sampled. A 10-second clip at 3-second intervals yields 0, 3, 6 and 9 — the last second is covered by nothing.
• The counter accumulates in floating point. After a few hundred steps, adding 0.1 repeatedly does not land on exactly 12.3, and the requested times drift by tiny amounts that are irrelevant for review and worth knowing about if you are reconciling a list of timestamps against a log.

The practical effect of the first point is that interval mode is biased towards the beginning of a clip. If you are sampling a two-hour recording at 60-second intervals to find where something changed, the change is just as likely to be at 1:59:30 as at 0:00:30, but your last sample is at 1:59:00 and the final minute is invisible. Add one timestamp at the end, or use count mode, which distributes the same number of samples across the whole clip.

This is the part of the interface that needs the most honest description, because the label is a promise the implementation cannot keep.

Keyframe mode in this tool steps through the clip at a fixed two-second interval. It does not read the container’s keyframe index. It does not know where the real keyframes are. It is a sampling grid with a name that suggests otherwise.

So what is the two-second grid actually for? A first pass over long footage where you want roughly one candidate per conversational beat, and where the alternative — scrubbing a two-hour timeline by hand — is much worse. It is a convenience, not an index. If you need the real keyframe positions, they are one command away:

ffprobe -v error -select_streams v:0 \
  -show_entries frame=key_frame,pts_time \
  -of csv=p=0 input.mp4 | awk -F, '$1 == 1 { print $2 }'

One trap in that command is worth a sentence, because it cost us a wrong answer. ffprobe prints fields in its own internal order, not in the order you list them: asking for pts_time,key_frame still yields key_frame first. An awk written against the order you asked for silently matches nothing and prints an empty list, which looks exactly like a file with no keyframes. Check the first line of output before trusting the filter.

The output is the ground truth the browser tool is approximating. On a typical export with a 2-second GOP the two lists will look similar. On a file with scene-cut keyframes — most professionally edited footage — they will not: the real keyframes land on cuts, which is far more useful for reviewing an edit than an arbitrary grid, and it is the reason a desktop tool that reads the index can find cut points in seconds.

The request is a seek, and a seek is not a lookup. Compressed video stores most frames as differences from other frames. To display the frame at 12.500 s the decoder must start at the nearest earlier keyframe and decode forward until it reaches the requested time. Everything between the keyframe and the target is decoded and discarded, which is why seeking in a long-GOP file is slower than seeking in a dense one, and why the cost does not scale with how far you jump but with how far you are from the previous keyframe.

Three things follow that matter for a still image.

1. The frame you get covers an interval, not an instant. At 30 fps each frame is visible for about 33 ms. Asking for 12.500 s gets you the frame whose presentation interval contains 12.500 s; asking for 12.505 s gets the same frame, and asking for 12.517 s gets the next one. The still cannot be more precise than the frame rate allows.
2. Browsers may adjust the reported time. After a seek, currentTime can be set to the actual position the decoder reached, which may differ from what you requested by a frame or so. Reading currentTime back after the seeked event is the only way to know what the browser thinks it is showing.
3. Variable frame rate files make the arithmetic non-obvious. In a screen recording that drops to 5 fps during a static stretch, one second of media time may contain five frames instead of thirty, and the relationship between a timestamp and a frame index is not linear.

None of this makes the exported still wrong. It makes it approximate at the level of one frame interval, which is fine for a thumbnail and not fine for a claim like “at 00:01:14.500 the dialog was already open”. For the second kind of claim, the next section gives you a way to check.

The only way to know how far off a seek lands is to make the frame number visible in the picture, extract from that copy, and read the number back. The obvious way to do this is to burn the digits in with drawtext and read them by eye. The better way is to burn a pattern a canvas can read, because then the check can run hundreds of times instead of four, and nobody has to squint at a screenshot.

# 20 s at 30 fps = 600 frames. Bar A encodes n mod 30, bar B encodes floor(n / 30).
# drawbox cannot do this: its x expression is evaluated once, so every frame is identical.
# geq re-evaluates per frame and has N, hence geq. Commas inside it must be escaped.
XA='(N-30*floor(N/30))*10.6'
ffmpeg -f lavfi -i "color=c=black:s=320x180:r=30:duration=20" \
  -vf "geq=lum='235*max(gt(Y\,19)*lt(Y\,45)*gt(X\,${XA}-1)*lt(X\,${XA}+7),gt(Y\,59)*lt(Y\,85)*gt(X\,floor(N/30)*16-1)*lt(X\,floor(N/30)*16+11))+16':cb=128:cr=128" \
  -c:v libx264 -preset veryfast -crf 20 -pix_fmt yuv420p \
  -g 250 -keyint_min 250 -sc_threshold 0 \
  counter.mp4

Two bars, 30 positions and 20 positions, which is 600 unambiguous frames. A canvas finds each bar’s centre column and multiplies back out; a person looking at the still sees which column is lit. Because the pattern is high-contrast and coarse, H.264 at a normal quality setting does not blur the bars into each other — the index survives the very compression the measurement is about.

We ran this against a 20-second, 30-frame-per-second clip (600 frames, keyframes every 250 frames, so at 0, 8.333 and 16.667 seconds) in Google Chrome for Testing 145.0.7632.6, headless, on macOS on Apple silicon. The file was served from localhost over HTTP with byte-range support, so seeking behaved like a remote file rather than a blob in memory. Each mode requested the times the tool would request, waited for seeked, drew the frame, and read the index back.

Zero difference in all 34 samples is the result, and it is worth being clear about what it means. Chrome does not snap to the nearest keyframe. It decodes forward from the previous one and presents the frame whose presentation interval contains the requested instant — at 18.182 s that meant frame 545, which is 45 frames past the keyframe at 500. The expensive part is real: the decoder did that work. It just did not lose accuracy doing it.

The practical consequence is a shift in where to look for error. If the seek is exact, a still that lands in the wrong place is almost always the arithmetic: a count mode asked for N points when the event sits between two of them, an interval that never sampled the final minute, or a timestamp computed from a 30 fps assumption applied to 29.97 fps footage — which accumulates about one frame of error per 33 seconds, and a full second by the end of a 33-minute recording.

Extraction accuracy has a second half that gets less attention: the frame was decoded correctly, and then it was written to a file. PNG is lossless and large; JPEG is lossy and small; the interesting question is how much smaller and how much lossy, on the two kinds of image people actually extract.

We encoded two synthetic frames with the browser’s own canvas.toBlob — one text-heavy frame with burned-in digits, one continuous-tone image with fine detail — and measured both the file size and the peak signal-to-noise ratio against the PNG original, inside the same page that produced the stills.

Read the ratios, not the decibels. On the photographic image JPEG is roughly an eighth of the PNG at the higher quality setting, and the loss is spread thinly across texture where nobody will find it. On the text image JPEG is a third of the PNG, and the loss is concentrated in the one place it is most visible: the edges of glyphs, which is exactly what makes a burned-in timestamp legible or illegible.

There is no best mode. There is a mode whose failure you can live with. Sort your task by what happens if the still lands a second away from the interesting moment.

The last row is not modesty. A browser seek is a best-effort operation on a stream it did not index, and no amount of interface polish changes that. The guide on evidence takes this further.