ffmpeg slow performance with hardware acceleration

Question

I have an Intel i7-8086k CPU with integrated graphics and trying to use ffmpeg to decode H264 to bmp frames for further processing over a UNIX pipe. I have ffmpeg 4.1.4 installed and latest vaapi libraries from fedora 30 repository (libva-2.4.1, libva-intel-driver-2.3.0).

I get really good performance using the benchmarking command (about 18x speed vs 3x without hwaccel):

ffmpeg  -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device /dev/dri/renderD128  -i file.mp4   -f null -
...
frame= 5459 fps=518 q=-0.0 Lsize=N/A time=00:03:01.71 bitrate=N/A speed=17.2x

However, when I add hwdownload filter to be able to convert the GPU output to BMP, the performance drops to ~ 2x speed (which is slower than doing the entire process in software--I also get some unexplained messages during decode about past duration being too large):

ffmpeg -v verbose  -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device /dev/dri/renderD128  -i file.mp4 -f image2pipe -vf hwdownload,format=yuv420p  -vcodec bmp pipe:1 > /dev/null

...
Past duration 0.989662 too large
...

** dropping frame 3172 from stream 0 at ts 317000:01:44.90 bitrate=3538957.0kbits/s dup=0 drop=3 speed=2.13x

This makes HW decode practically useless for my application--I need the entire decode to take less than 20 secs, which is the case in the benchmark command (~about 10 secs).

My question is there anything in the command line above that is making the process slower ? (I tried different pixel formats, with no improvement).

Thanks.