ProRes transcoding in ffmpeg too slow

1

We are trying to transcode movies / clips into ProRes LT by using following options:

ffmpeg -threads "16" -i "$2" -flags +ildct -vcodec prores -profile 1 -qscale 4 \
-threads "16" -r 24 -vf "yadif=0:-1:-1, scale=1920:1080" -aspect 16:9 \
-acodec pcm_s32le

The thing is we have a transcoding server which only does transcodings and it has 16 threads – only one is used. FFmpeg starts with 14 fps and then drops down with a transcoding rate of around 8 fps:

frame=    9 fps=0.0 q=0.0 size=    2647kB time=00:00:00.30 bitrate=72271.8kbits/s dup=1 drop=0    
frame=   15 fps= 14 q=0.0 size=    5623kB time=00:00:00.50 bitrate=92120.9kbits/s dup=1 drop=0    
frame=   20 fps= 13 q=0.0 size=    8268kB time=00:00:00.66 bitrate=101600.1kbits/s dup=1 drop=0    
frame=   25 fps= 12 q=0.0 size=   10964kB time=00:00:00.83 bitrate=107785.2kbits/s dup=1 drop=0    
frame=   30 fps= 11 q=0.0 size=   13678kB time=00:00:01.00 bitrate=112052.5kbits/s dup=1 drop=0    
frame=   35 fps= 10 q=0.0 size=   16407kB time=00:00:01.16 bitrate=115204.0kbits/s dup=1 drop=0    
frame=   40 fps= 10 q=0.0 size=   19131kB time=00:00:01.33 bitrate=117539.2kbits/s dup=1 drop=0    
frame=   45 fps=9.8 q=0.0 size=   21842kB time=00:00:01.50 bitrate=119286.8kbits/s dup=1 drop=0    
frame=   50 fps=9.6 q=0.0 size=   24572kB time=00:00:01.66 bitrate=120776.7kbits/s dup=1 drop=0    
frame=   54 fps=9.4 q=0.0 size=   26755kB time=00:00:01.80 bitrate=121767.1kbits/s dup=1 drop=0    
frame=   58 fps=9.3 q=0.0 size=   28946kB time=00:00:01.93 bitrate=122653.0kbits/s dup=1 drop=0    
frame=   62 fps=9.2 q=0.0 size=   31150kB time=00:00:02.06 bitrate=123474.0kbits/s dup=1 drop=0    
frame=   66 fps=9.1 q=0.0 size=   33352kB time=00:00:02.20 bitrate=124189.0kbits/s dup=1 drop=0    
frame=   70 fps=9.0 q=0.0 size=   35569kB time=00:00:02.33 bitrate=124877.8kbits/s dup=1 drop=0    
frame=   74 fps=8.9 q=0.0 size=   37787kB time=00:00:02.46 bitrate=125494.2kbits/s dup=1 drop=0    
frame=   78 fps=8.8 q=0.0 size=   40010kB time=00:00:02.60 bitrate=126062.6kbits/s dup=1 drop=0    
frame=   82 fps=8.7 q=0.0 size=   42229kB time=00:00:02.73 bitrate=126563.9kbits/s dup=1 drop=0    
frame=   86 fps=8.6 q=0.0 size=   44434kB time=00:00:02.86 bitrate=126979.3kbits/s dup=1 drop=0    
frame=   90 fps=8.6 q=0.0 size=   46641kB time=00:00:03.00 bitrate=127361.4kbits/s dup=1 drop=0    
frame=   94 fps=8.5 q=0.0 size=   48846kB time=00:00:03.13 bitrate=127706.8kbits/s dup=1 drop=0    
frame=   98 fps=8.5 q=0.0 size=   51046kB time=00:00:03.26 bitrate=128010.0kbits/s dup=1 drop=0    
frame=  101 fps=8.3 q=0.0 size=   52701kB time=00:00:03.36 bitrate=128235.3kbits/s dup=1 drop=0    
...

We have searched with google but we didn't find any working solution for this problem, does anyone of you guys have experienced the same problem? And if you did, how did you solve it in the end?

@LordNeckbeard Here are the first and the last 50 lines of transcoding:

ffmpeg version 0.10.2.git Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 16 2013 15:47:31 with gcc 4.1.2 20080704 (Red Hat 4.1.2-51)
  configuration: --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvpx --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --disable-ffplay --enable-shared --enable-gpl --enable-postproc --enable-nonfree --enable-avfilter --enable-pthreads --extra-cflags=-fPIC --enable-libfreetype
  libavutil      51. 55.100 / 51. 55.100
  libavcodec     54. 23.100 / 54. 23.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 77.100 /  2. 77.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[h264 @ 0x1a574240] Increasing reorder buffer to 1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/content/flowcenter/highres/dokfest/ftp_upload_20131108_2/Cacheu2.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2012-10-17 05:46:57
LOGGER: set frames to 14950
  Duration: 00:09:58.00, start: 0.000000, bitrate: 24157 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080, 23863 kb/s, 25 fps, 25 tbr, 2500 tbn, 5k tbc
    Metadata:
      creation_time   : 2012-10-17 08:24:55
      handler_name    : Apple Video Media Handler
    Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, s16, 289 kb/s
    Metadata:
      creation_time   : 2012-10-17 08:25:00
      handler_name    : Apple Sound Media Handler
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74)
    Metadata:
      creation_time   : 2012-10-17 08:27:50
      handler_name    : Time Code Media Handler
      timecode        : 01:00:00:00
Please use -profile:a or -profile:v, -profile is ambiguous
Please use -q:a or -q:v, -qscale is ambiguous
[yadif @ 0x1a5a6e40] mode:0 parity:-1 auto_enable:-1
[buffer @ 0x1a5afb00] w:1920 h:1080 pixfmt:yuv420p tb:1/2500 sar:0/1 sws_param:flags=2
[buffersink @ 0x1a5afa80] No opaque field provided
[scale @ 0x1a5afc60] w:1920 h:1080 fmt:yuv420p sar:0/1 -> w:1920 h:1080 fmt:yuv422p10le sar:0/1 flags:0x4
[aformat @ 0x1a665fa0] auto-inserting filter 'auto-inserted resampler 0' between the filter 'Parsed_anull_0' and the filter 'aformat'
[aresample @ 0x1a5daf00] chl:stereo fmt:s16 r:48000Hz -> chl:stereo fmt:s32 r:48000Hz
Output #0, mov, to '/content/flowcenter/other/Cacheu2_2_1.mov.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2012-10-17 05:46:57
    encoder         : Lavf54.6.101
    Stream #0:0(eng): Video: prores (apcs) (apcs / 0x73637061), yuv422p10le, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 tbn, 25 tbc
    Metadata:
      creation_time   : 2012-10-17 08:24:55
      handler_name    : Apple Video Media Handler
    Stream #0:1(eng): Audio: pcm_s32le (in32 / 0x32336E69), 48000 Hz, stereo, s32, 3072 kb/s
    Metadata:
      creation_time   : 2012-10-17 08:25:00
      handler_name    : Apple Sound Media Handler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> prores)
  Stream #0:1 -> #0:1 (aac -> pcm_s32le)
video:7615955kB audio:224272kB global headers:0kB muxing overhead 0.004298%
Executing:
"/home/mfc/ffmpeg/one_for_all.mov.sh" 1281038 "/content/flowcenter/highres/dokfest/ftp_upload_20131108_2/Cacheu2.mov" "/content/flowcenter/other/Cacheu2_2_1.mov.mov"
1c
ffmpeg -threads 16 -i /content/flowcenter/highres/dokfest/ftp_upload_20131108_2/Cacheu2.mov -flags +ildct -vcodec prores -profile 1 -qscale 4 -threads 16 -vf yadif=0:-1:-1, scale=1920:1080 -aspect 16:9 -acodec pcm_s32le /content/flowcenter/other/Cacheu2_2_1.mov.mov
needed 1679 seconds

Anyways I think it would be way too long to post the whole thing. I registered a slight change, when I transcoded it again, because I could not find the logfile from yesterdays clip.

@Rajib sorry that was my fault with using -threads two times, I just thought maybe I had put the threads command at another place in the script so it would actually use the command, but it didn't work so I deleted it again. As for the command -i "$2" our software works with a mix of ffmpeg commands and variables and as far as I understood $2 is another way to describes the path to our input file so we don't have to use xyz.mov anymore. And yes we want a 1080p quality even it that means we have to upscale the clip.

sebastian

Posted 2013-12-11T15:39:01.510

Reputation: 141

1Please show the complete ffmpeg console output (or at least the first and last ~50 lines). – llogan – 2013-12-11T17:17:14.183

Out of curiosity, why do you need to put in -threads "16" twice? Is $2 a sequence of images or a mov file? Are you scaling to 1920x1080 from a greater size? – Rajib – 2013-12-11T19:12:34.467

LordNeckbeard asked for first 50 lines of output of ffmpeg command, not of transcoding process, – ptQa – 2013-12-12T09:08:05.377

Complete console output, first 50 and last 50 lines, or complete output with multiple repeating lines trimmed out are all acceptable. – llogan – 2013-12-12T19:26:58.633

You can start with a minimal command and gradually include options to see if you find a bottleneck in the process: time ffmpeg -y -i input -c:v prores -t 60 -an -f null - – llogan – 2013-12-12T23:35:50.267

No answers