Reduce video frame rate without dropping useful frames

Question

I have a video from a security camera that was originally recorded at a low frame rate, maybe 15 fps, maybe 10 fps, maybe even less. It has since been converted by someone else to 25 fps (without changing the duration). I assume that extra, duplicate frames have been added, and possibly this has also slightly distorted the exact time at which the original frames are displayed.

I want to re-encode the video with ffmpeg to its original frame rate, without dropping any of the useful frames of actual motion. If I just use the fps filter it won't be selective about which frames it keeps, and I assume that due to rounding errors it could end up keeping some of the duplicate frames and permanently losing useful frames, making things worse. In any case, I don't know what number to tell the filter because I don't know exactly what the original frame rate was!

How can I proceed to repair this video with ffmpeg?

ffmpeg -i "orig.mp4" -an -vcodec h264 -vf "fps=???" "fixed.mp4"

Answer 1

I managed to fix my file. I found the mpdecimate filter, which drops duplicate or near duplicate frames from a video stream. First I tried:

ffmpeg -loglevel debug -i orig.mp4 -an -vf "mpdecimate" test.mp4

I added -loglevel debug to get more info during transcoding. The info showed what mpdecimate was doing with the frames, which was lots and lots of this pattern:

lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:44032 pts_time:3.44 drop_count:1
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:44544 pts_time:3.48 drop_count:2
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:45056 pts_time:3.52 drop_count:3
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:45568 pts_time:3.56 drop_count:4
781>=hi keep pts:46080 pts_time:3.6 drop_count:-1
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:46592 pts_time:3.64 drop_count:1
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:47104 pts_time:3.68 drop_count:2
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:47616 pts_time:3.72 drop_count:3
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:48128 pts_time:3.76 drop_count:4
821>=hi keep pts:48640 pts_time:3.8 drop_count:-1
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:49152 pts_time:3.84 drop_count:1
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:49664 pts_time:3.88 drop_count:2
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:50176 pts_time:3.92 drop_count:3
lo:0<217 lo:0<-2147483648 lo:0<-2147483648 drop pts:50688 pts_time:3.96 drop_count:4
793>=hi keep pts:51200 pts_time:4 drop_count:-1

I.e., it was consistently dropping 4 frames in a row as duplicates and keeping every 5th. This showed that the original frame rate was 1/5th of what it was now. I was lucky! Since the video's current 25 fps was a multiple of the original frame rate, there shouldn't be a problem with temporal misalignment of the kept and dropped frames. Therefore I retranscoded the original video with nothing more complex than -vf "fps=5", and the output seemed to be fine. Just to be sure, I piped the output through the mpdecimate filter, and it detected all remaining frames as a "keep". So the original file was not so messed up as I thought.

Answer 2

If the extra frames are duplicates and not interpolations, then the following may work:

ffmpeg -i orig.mp4 -an -vf "select='gt(scene\,0.001)',setpts=N/(10*TB)" -r 10 fixed.mp4

The idea is to select all frames not a duplicate of the preceding frame. Then PTSes are regenerated as per the output rate specified. You may have to tweak the scene value up or down to get the detection filter right. If the video appears faster or slower than real-time then lower or raise the rate (and setpts denominator value) respectively.