Compensate key-frame time distance when video cutting with ffmpeg

1

1

My requirement is to frame accureately(few frames up/down) cut on the fly mp4(h264 + aac) https video stream with ffmpeg on mobile phone fastest as I can - meaning I would trade if necessary bad compression(bigger output file) for greater cutting speed. Currently am cutting the video with this command:

ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 00:00:10.000 -c:a copy -c:v copy out.mp4

This approach(bit stream copying) is very fast(less than a second on good network for 10sec long video cut) but is not accurately cutting the video. Video is being cut on key-frames which is imprecise.

So my idea is to compensate this imprecision(distance from key-frame) by seeking on video before playing it for:

delta = start_time - first_key_frame_timestamp_before_start_time

To calculate delta i need a very fast way to get time_stamp of first key-frame before start-time. I've seen post which explains how to get timestamp of first key-frame before start_time but this approach is not practical for long(2hours+) videos because it would take to long for this operation to reach lets say middle of video. I would need something like this:

ffmpeg -ss 01:30:06.000 -i https://domain/in.mp4 -t 2 -show_frames > frame_meta.txt

I have also tried doing precise cut by re-encoding the video(2hours long video, 10sec long video cut in the middle of the video) with this command:

ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 10 out.mp4

but it takes to long(~16seconds) on mobile to finish compared to less than a second bit-stream copy command. Thx for any useful hint or solution.

EDIT 1:

I managed to calculate delta and get timestamp of first keyFrame before start_time by using ExoPlayer(Android multimedia player) internal classes and that approach is very fast 500ms(very short videos) up to 1500ms(for very long videos). Essentially this approach is based on parsing meta_headers(moov atom) of remote mp4 stream to found adequate timeStamp of keyFrame before cut start_time. Now NEW PROBLEM arouse, when video is cut with:

ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 00:00:10.000 -c:a copy -c:v copy out.mp4

edit-list atom is added to meta header as understood from this and this post which is there to tell player to start play with delta time-offset to compensate imprecise video cutting which is done with ffmpeg bitStream copy command. But players I use on Android does not support edit-list playbacks because they just start playing normally from the beginning of video or are freezed on some frame for few moments while audio is playing and then they continue playing normally.

When I try to seek for previously calculated delta and then start play on cutted video, am not positioned on correct frame (eg. calculated delta=1 381ms but real/working delta=444ms) and my plan to compensate imprecise cutting is not working :(. If you are asking yourself if I have correctly calculated delta I've checked it with IsoViewer(mp4 container atom parser) and mannualy by stepping through frames with Avidemux and looking into time_stamp/frame_type value pair...

xyman

Posted 2016-10-11T15:55:25.217

Reputation: 11

I think this is still the fastest method.

– Gyan – 2016-10-11T16:32:37.357

Thx for good tip, I've took a look at your answer. But how would i know which key-frame index in STSS atom is the one before start_time because that atom only holds indexes of key frames - if it was a key-value pair like [{keyframe_index, time_stamp}, {}...] it would be very easy? – xyman – 2016-10-11T17:00:44.980

The index for KF will be time x FPS, so 0x30 in a 25 fps video indicates a time of 1.96s. This assumes a constant frame rate video. – Gyan – 2016-10-11T17:04:14.677

Oh i see, perfect. But the other issue would be how to feed mp4parser with https stream because as am familiar with mp4parser its only working with local full length video files ? Am open to any kind of hacks to make ti work. – xyman – 2016-10-11T17:13:09.323

@Mulvya Hi, I managed to quickly parse moov atom with ExoPlayer(android video player) internal classes and get timestamp of first keyFrame before cut time. Now i have other issue, when cutting video with a/v codec copy ffmpeg command, edit-list atom is created which as I understood is put there to compensate imprecise video cutting - please correct me if am wrong. But when edit list is present there is no need to compensate imprecise cutting, video player should just play from point which is defined in edit list? But most players does not support edit lists ... – xyman – 2016-10-21T10:31:37.707

@Mulvya I have expanded my question with result of investigation and related problem that arouse in the mean time. – xyman – 2016-10-21T12:38:40.740

No answers