experiment:encoding-quality

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
experiment:encoding-quality [2020/06/11 22:39] – [Table] ischluffexperiment:encoding-quality [2021/04/15 13:36] (current) – ischluff
Line 1: Line 1:
 ====== Encoding-Quality-Testing ====== ====== Encoding-Quality-Testing ======
-Created with the [[https://bitbucket.fem.tu-ilmenau.de/projects/BROADCAST/repos/vmaf-docker/browse|vmaf-docker]] repo.+Created with the [[https://github.com/voc/voctoquality|voctoquality]] repo.
  
-==== Testing VAAPI vs. software encoding for live streaming ==== +===== Reference content ===== 
-**The graphs currently contain no confidence intervals (values may differ, but the difference may not be relevant)**+Reference content is either our own (https://media.ccc.de) or from Xiph (https://media.xiph.org/video/derf/).
  
-The codec settings are the ones currently in use by the C3VOC for live streaming.+The specific sources are found in https://github.com/voc/voctoquality/blob/master/data/sources.json
  
-All the reference content is 1080p25.+All the reference content is converted to 8bit 1080p25.
  
-=== Codec Settings === +===== Software ===== 
-  * vaapi: nothing but keyint and bitrates set +  * ffmpeg-4.1.4 (same as in buster) 
-<code> +  * libvmaf-1.3.15 
-    -vaapi_device /dev/dri/renderD128 +  * libvpx-1.8.2 (bullseye) 
-    -hwaccel vaapi -hwaccel_output_format vaapi +  * libx264-2:0.155.2917+git0a84d98-2 (bullseye) 
-    -i $ref + 
-    -vf 'format=nv12|vaapi,hwupload' +===== Testing VAAPI vs. software encoding for live streaming ===== 
-    -c:v {codec}_vaapi +The codec settings are the ones currently in use by the C3VOC for live streaming. 
-    -keyint_min:v 75 -g:v 75 + 
-    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k +See https://github.com/voc/voctoquality/blob/master/data/profiles/voc_streaming.py for the specific ffmpeg options.
-</code> +
-  * libvpx-vp9 (current C3VOC transcoding settings) +
-<code> +
-    -i $ref +
-    -c:v libvpx-vp9 +
-    -deadline:v realtime -cpu-used:v 8 +
-    -threads:v 8 +
-    -frame-parallel:v 1 -tile-columns:v 2 +
-    -keyint_min:v 75 -g:v 75 +
-    -crf:v 23 +
-    -b:v {rate}k -maxrate:v {rate}k -bufsize {rate}k +
-</code> +
-  * x264 (current C3VOC encoder-stream settings) +
-<code> +
-    -i $ref +
-    -c:v libx264 -preset:v veryfast +
-    -profile:v main -flags +cgop +
-    -threads:v 0 -g:v 75 +
-    -crf:v 21 +
-    -maxrate:v {rate}k -bufsize {rate}k +
-</code>+
  
 The codecs are tested with bitrates from 1Mbit/s to 5.2Mbit/s. The codecs are tested with bitrates from 1Mbit/s to 5.2Mbit/s.
-Currently the 1080p25 streams are coded with 2.8Mbit/s maxrate. 
  
 +==== Graphs ====
 +{{ :experiment:streaming_rates.pdf |}}
 +{{ :experiment:streaming_refs.pdf |}}
 +{{ :experiment:streaming_speeds.pdf |}}
 +
 +==== Comments ====
 +
 +^ Platform                                         ^ Label        ^ Hardware Codecs    ^ Comments                                                                                                                                                        ^
 +^ Intel 6th Gen (i7-6600U)                         | skylake      | h.264, h.265       | h.264 scores better than our VP9 and x264 streaming settings and is pretty fast\\ h.265 even scores a little bit higher but is considerably slower              |
 +^ Intel 8th Gen (i7-8665U)                         | whiskeylake  | h.264, h.265, VP9  | h.264 and h.265 results worse but faster than skylake, have to check whats up with that\\ VP9 scores worse than H.264 and has broken rate-control for <2mbit/s  |
 +^ AMD Navi (RX 5700 XT) + Ryzen 2nd Gen (R5-2600)  | navi10       | h.264, h.265       | h.264 is worse than x264 on veryfast...\\ h.265 produces 1920x1088 output?                                                                                      |
 +^ Nvidia Turing (RTX2080 Super)                    | turing       | h.264, h.265       | nvidia-driver-460, cuda-toolkit 9.1.85\\ best h.265 encode, h.264 behind Intel                                                                                  |
 +
 +
 +==== Finding a Tracker VP9 Profile ====
 +=== HD-Profile ===
 +
 +Goals: 
 +  * improve quality over previous libvpx (VP8) profile
 +  * don't increase encoding time too much
 +  * smaller size but similar quality as x264 master
  
-=== Versions === 
-  * CPU: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz 
-  * ffmpeg-4.1.4 
-  * libvmaf-1.3.15 
-  * libvpx-1.7.0 (buster) 
-  * libx264-2:0.155.2917+git0a84d98-2 (buster) 
  
-=== Reference === +Notes: 
-Xiph Content is found on https://media.xiph.org/video/derf/ +  * realtime + speed <= 3: slower and worse than quality good
-  * Ref1 - Noisy MPEG2 36C3 +
-  * Ref2 - MPEG2 36C3 +
-  * Red Kayak - Xiph (converted to 8Bit 25FPS) +
-  * Rush Hour - Xiph+
  
-=== Results ===+``` 
 +-i $ref 
 +-c:v libvpx-vp9 
 +-quality:v good 
 +-crf:v 30 
 +-b:v 6000k 
 +-maxrate:v 8000k 
 +-minrate:v 2000k 
 +-row-mt 1 
 +-frame-parallel:v 1 -tile-columns:v 2 
 +```
  
-^ Reference  ^ Intel 8th Gen (i7-8665U)                                                                                                                                                                                                                                                                                                                                                                                                          ^ AMD Navi (RX 5700 XT) + AMD Ryzen 5 2600\\ {{ :experiment:navi_scores.json }}                       ^ +=== SD-Profile === 
-| Ref1       | {{ :experiment:bitrates_ref1.pdf }}\\ {{ :experiment:speed_ref1.pdf }}\\   * no improvement beyond 2Mbit/s\\   * VAAPI HEVC & h.264 with best quality\\   * libvpx-VP9 better and faster than x264\\   * VAAPI HEVC not faster than software encoding                                                                                                                                                                             | {{ :experiment:navi_bitrates_ref1.nut.pdf }}\\ {{ :experiment:navi_speed_ref1.nut.pdf }}            | +  * worse parallelization and therefore speed than HD-profile 
-| Ref2       | {{ :experiment:bitrates_ref2.pdf }}\\ {{ :experiment:speed_ref2.pdf }}\\   * no improvement beyond 1.5Mbit/s\\   * similar Mean for all codecs\\   * better Minimum scores for libvpx/x264\\   * high speeds for libvpx/x264                                                                                                                                                                                                      | {{ :experiment:navi_bitrates_ref2.nut.pdf }}\\ {{ :experiment:navi_speed_ref2.nut.pdf }}            | +  * worse rate-control than HD-profile (doesn't reach target bitrate) 
-| Red Kayak  | {{ :experiment:bitrates_red_kayak.pdf }}\\ {{ :experiment:speed_red_kayak.pdf }}\\   * very good quality never reached\\   * VAAPI VP9 has rate control problems at low bitrates (resulting bitrate large but quality low)\\   * VAAPI HEVC > VAAPI h.264 > libvpx-VP9 at high bitrates\\   * libvpx-VP9 best at low bitrates\\   * libvpx-VP9 better than VAAPI VP9, albeit slower\\   * libvpx-VP9 better and faster than x264  | {{ :experiment:navi_bitrates_red_kayak.nut.pdf }}\\ {{ :experiment:navi_speed_red_kayak.nut.pdf }}  | +  * bufsize doesn't change anything 
-| Rush Hour  | {{ :experiment:bitrates_rush_hour.pdf }}\\ {{ :experiment:speed_rush_hour.pdf }}\\  VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264                                                                                                                                                                                                                                                                                      | {{ :experiment:navi_bitrates_rush_hour.nut.pdf }}\\ {{ :experiment:navi_speed_rush_hour.nut.pdf }}  |+  * no threads parallelizes best 
 +  * frame-parallel doesn't work (frame too small?) 
 +  * bitrate with crf is more like maxrate
  
-=== Intel Conclusion === 
-Keep in mind that this only applies to the stream-settings used 
-  * Quality: VAAPI HEVC > VAAPI h.264 > libvpx-VP9 > VAAPI-VP9 > x264 
-  * Speed: VAAPI h.264 > VAAPI VP9 > VAAPI HEVC >= libvpx-VP9 ~= x264 
-  * VAAPI VP9 has lower speeds and quality than VAAPI h.264 
-  * libvpx-VP9 has higher quality than x264 at the same speeds 
-  * VAAPI-HEVC outperforms VAAPI-h.264 on hard content, but is 3x slower 
  
  • experiment/encoding-quality.txt
  • Last modified: 2021/04/15 13:36
  • by ischluff