experiment:encoding-quality

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Last revisionBoth sides next revision
experiment:encoding-quality [2020/06/11 22:39] – [Table] ischluffexperiment:encoding-quality [2021/04/15 13:36] – [Testing VAAPI vs. software encoding for live streaming] ischluff
Line 1: Line 1:
 ====== Encoding-Quality-Testing ====== ====== Encoding-Quality-Testing ======
-Created with the [[https://bitbucket.fem.tu-ilmenau.de/projects/BROADCAST/repos/vmaf-docker/browse|vmaf-docker]] repo.+Created with the [[https://github.com/voc/voctoquality|voctoquality]] repo.
  
-==== Testing VAAPI vs. software encoding for live streaming ==== +===== Reference content ===== 
-**The graphs currently contain no confidence intervals (values may differ, but the difference may not be relevant)**+Reference content is either our own (https://media.ccc.de) or from Xiph (https://media.xiph.org/video/derf/).
  
-The codec settings are the ones currently in use by the C3VOC for live streaming.+The specific sources are found in https://github.com/voc/voctoquality/blob/master/data/profiles/voc_streaming.py
  
-All the reference content is 1080p25.+All the reference content is converted to 8bit 1080p25.
  
-=== Codec Settings === +===== Software ===== 
-  * vaapi: nothing but keyint and bitrates set +  * ffmpeg-4.1.4 (same as in buster) 
-<code> +  * libvmaf-1.3.15 
-    -vaapi_device /dev/dri/renderD128 +  * libvpx-1.8.2 (bullseye) 
-    -hwaccel vaapi -hwaccel_output_format vaapi +  * libx264-2:0.155.2917+git0a84d98-2 (bullseye) 
-    -i $ref + 
-    -vf 'format=nv12|vaapi,hwupload' +===== Testing VAAPI vs. software encoding for live streaming ===== 
-    -c:v {codec}_vaapi +The codec settings are the ones currently in use by the C3VOC for live streaming. 
-    -keyint_min:v 75 -g:v 75 + 
-    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k +See https://github.com/voc/voctoquality/blob/master/data/profiles/voc_streaming.py for the specific ffmpeg options.
-</code> +
-  * libvpx-vp9 (current C3VOC transcoding settings) +
-<code> +
-    -i $ref +
-    -c:v libvpx-vp9 +
-    -deadline:v realtime -cpu-used:v 8 +
-    -threads:v 8 +
-    -frame-parallel:v 1 -tile-columns:v 2 +
-    -keyint_min:v 75 -g:v 75 +
-    -crf:v 23 +
-    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k +
-</code> +
-  * x264 (current C3VOC encoder-stream settings) +
-<code> +
-    -i $ref +
-    -c:v libx264 -preset:v veryfast +
-    -profile:v main -flags +cgop +
-    -threads:v 0 -g:v 75 +
-    -crf:v 21 +
-    -maxrate:v {rate}k -bufsize {rate}k +
-</code>+
  
 The codecs are tested with bitrates from 1Mbit/s to 5.2Mbit/s. The codecs are tested with bitrates from 1Mbit/s to 5.2Mbit/s.
-Currently the 1080p25 streams are coded with 2.8Mbit/s maxrate. 
  
 +==== Graphs ====
 +{{ :experiment:streaming_rates.pdf |}}
 +{{ :experiment:streaming_refs.pdf |}}
 +{{ :experiment:streaming_speeds.pdf |}}
 +
 +==== Comments ====
 +
 +^ Platform                                         ^ Label        ^ Hardware Codecs    ^ Comments                                                                                                                                                        ^
 +^ Intel 6th Gen (i7-6600U)                         | skylake      | h.264, h.265       | h.264 scores better than our VP9 and x264 streaming settings and is pretty fast\\ h.265 even scores a little bit higher but is considerably slower              |
 +^ Intel 8th Gen (i7-8665U)                         | whiskeylake  | h.264, h.265, VP9  | h.264 and h.265 results worse but faster than skylake, have to check whats up with that\\ VP9 scores worse than H.264 and has broken rate-control for <2mbit/s  |
 +^ AMD Navi (RX 5700 XT) + Ryzen 2nd Gen (R5-2600)  | navi10       | h.264, h.265       | h.264 is worse than x264 on veryfast...\\ h.265 produces 1920x1088 output?                                                                                      |
 +^ Nvidia Turing (RTX2080 Super)                    | turing       | h.264, h.265       | nvidia-driver-460, cuda-toolkit 9.1.85\\ best h.265 encode, h.264 behind Intel                                                                                  |
 +
 +
 +==== Finding a Tracker VP9 Profile ====
 +=== HD-Profile ===
 +
 +Goals: 
 +  * improve quality over previous libvpx (VP8) profile
 +  * don't increase encoding time too much
 +  * smaller size but similar quality as x264 master
  
-=== Versions === 
-  * CPU: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz 
-  * ffmpeg-4.1.4 
-  * libvmaf-1.3.15 
-  * libvpx-1.7.0 (buster) 
-  * libx264-2:0.155.2917+git0a84d98-2 (buster) 
  
-=== Reference === +Notes: 
-Xiph Content is found on https://media.xiph.org/video/derf/ +  * realtime + speed <= 3: slower and worse than quality good
-  * Ref1 - Noisy MPEG2 36C3 +
-  * Ref2 - MPEG2 36C3 +
-  * Red Kayak - Xiph (converted to 8Bit 25FPS) +
-  * Rush Hour - Xiph+
  
-=== Results ===+``` 
 +-i $ref 
 +-c:v libvpx-vp9 
 +-quality:v good 
 +-crf:v 30 
 +-b:v 6000k 
 +-maxrate:v 8000k 
 +-minrate:v 2000k 
 +-row-mt 1 
 +-frame-parallel:v 1 -tile-columns:v 2 
 +```
  
-^ Reference  ^ Intel 8th Gen (i7-8665U)                                                                                                                                                                                                                                                                                                                                                                                                          ^ AMD Navi (RX 5700 XT) + AMD Ryzen 5 2600\\ {{ :experiment:navi_scores.json }}                       ^ +=== SD-Profile === 
-| Ref1       | {{ :experiment:bitrates_ref1.pdf }}\\ {{ :experiment:speed_ref1.pdf }}\\   * no improvement beyond 2Mbit/s\\   * VAAPI HEVC & h.264 with best quality\\   * libvpx-VP9 better and faster than x264\\   * VAAPI HEVC not faster than software encoding                                                                                                                                                                             | {{ :experiment:navi_bitrates_ref1.nut.pdf }}\\ {{ :experiment:navi_speed_ref1.nut.pdf }}            | +  * worse parallelization and therefore speed than HD-profile 
-| Ref2       | {{ :experiment:bitrates_ref2.pdf }}\\ {{ :experiment:speed_ref2.pdf }}\\   * no improvement beyond 1.5Mbit/s\\   * similar Mean for all codecs\\   * better Minimum scores for libvpx/x264\\   * high speeds for libvpx/x264                                                                                                                                                                                                      | {{ :experiment:navi_bitrates_ref2.nut.pdf }}\\ {{ :experiment:navi_speed_ref2.nut.pdf }}            | +  * worse rate-control than HD-profile (doesn't reach target bitrate) 
-| Red Kayak  | {{ :experiment:bitrates_red_kayak.pdf }}\\ {{ :experiment:speed_red_kayak.pdf }}\\   * very good quality never reached\\   * VAAPI VP9 has rate control problems at low bitrates (resulting bitrate large but quality low)\\   * VAAPI HEVC > VAAPI h.264 > libvpx-VP9 at high bitrates\\   * libvpx-VP9 best at low bitrates\\   * libvpx-VP9 better than VAAPI VP9, albeit slower\\   * libvpx-VP9 better and faster than x264  | {{ :experiment:navi_bitrates_red_kayak.nut.pdf }}\\ {{ :experiment:navi_speed_red_kayak.nut.pdf }}  | +  * bufsize doesn't change anything 
-| Rush Hour  | {{ :experiment:bitrates_rush_hour.pdf }}\\ {{ :experiment:speed_rush_hour.pdf }}\\  VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264                                                                                                                                                                                                                                                                                      | {{ :experiment:navi_bitrates_rush_hour.nut.pdf }}\\ {{ :experiment:navi_speed_rush_hour.nut.pdf }}  |+  * no threads parallelizes best 
 +  * frame-parallel doesn't work (frame too small?) 
 +  * bitrate with crf is more like maxrate
  
-=== Intel Conclusion === 
-Keep in mind that this only applies to the stream-settings used 
-  * Quality: VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264 
-  * Speed: VAAPI h.264 > VAAPI VP9 > VAAPI HEVC >= libvpx-VP9 ~= x264 
-  * VAAPI VP9 has lower speeds and quality than VAAPI h.264 
-  * libvpx-VP9 has higher quality than x264 at the same speeds 
-  * VAAPI-HEVC outperforms VAAPI-h.264 on hard content, but is 3x slower 
  
  • experiment/encoding-quality.txt
  • Last modified: 2021/04/15 13:36
  • by ischluff