Encoding-Quality-Testing

Created with vmaf-docker repo.

Currently no confidence intervals (values may differ, but the difference may not be relevant)

All the content is 1080p25.

• vaapi: nothing but keyint and bitrates set

    -vaapi_device /dev/dri/renderD128
    -hwaccel vaapi -hwaccel_output_format vaapi
    -i $ref
    -vf 'format=nv12|vaapi,hwupload'
    -c:v {codec}_vaapi
    -keyint_min:v 75 -g:v 75
    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k

• libvpx-vp9 (current C3VOC transcoding settings)

    -i $ref
    -c:v libvpx-vp9
    -deadline:v realtime -cpu-used:v 8
    -threads:v 8
    -frame-parallel:v 1 -tile-columns:v 2
    -keyint_min:v 75 -g:v 75
    -crf:v 23
    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k

• x264 (current C3VOC encoder-stream settings)

    -i $ref
    -c:v libx264 -preset:v veryfast
    -profile:v main -flags +cgop
    -threads:v 0 -g:v 75
    -crf:v 21
    -maxrate:v {rate}k -bufsize {rate}k

The codecs are tested with bitrates from 1Mbit/s to 5.2Mbit/s. Currently the 1080p25 streams are coded with 2.8Mbit/s maxrate.

• CPU: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
• ffmpeg-4.1.4
• libvmaf-1.3.15
• libvpx-1.7.0 (buster)
• libx264-2:0.155.2917+git0a84d98-2 (buster)

bitrates_ref1.pdf

speed_ref1.pdf

• no improvement beyond 2Mbit/s
• VAAPI HEVC & h.264 with best quality
• libvpx-VP9 better and faster than x264
• VAAPI HEVC not faster than software encoding

bitrates_ref2.pdf

speed_ref2.pdf

• no improvement beyond 1.5Mbit/s
• similar Mean for all codecs
• better Minimum scores for libvpx/x264
• high speeds for libvpx/x264

bitrates_red_kayak.pdf

speed_red_kayak.pdf

• very good quality never reached
• VAAPI VP9 has rate control problems at low bitrates (resulting bitrate large but quality low)
• VAAPI HEVC > VAAPI h.264 > libvpx-VP9 at high bitrates
• libvpx-VP9 best at low bitrates
• libvpx-VP9 better than VAAPI VP9, albeit slower
• libvpx-VP9 better and faster than x264

bitrates_rush_hour.pdf

speed_rush_hour.pdf

• VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264

Keep in mind that this only applies to the stream-settings used

• Quality: VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264
• Speed: VAAPI h.264 > VAAPI VP9 > VAAPI HEVC >= libvpx-VP9 ~= x264
• VAAPI VP9 has lower speeds and quality than VAAPI h.264
• libvpx-VP9 has higher quality than x264 at the same speeds
• VAAPI-HEVC outperforms VAAPI-h.264 on hard content, but is 3x slower