= Video Acceleration

https://en.wikipedia.org/wiki/Category:Video_acceleration

https://en.wikipedia.org/wiki/Category:Video_compression_and_decompression_ASIC

= APIs

== VAAPI VAAPI was specified by intel but can be used with hardware of different vendors. It is well supported in gstreamer and ffmpeg. In most cases this is you best option. Only for some hardware / driver combinations you may get better results with one of the more proprietary APIs / drivers.

https://01.org/linuxgraphics/community/vaapi

https://freedesktop.org/wiki/Software/vaapi/

The tool vainfo from the libva-utils repository ( https://github.com/intel/libva-utils/tree/master/vainfo ) can give you an overview what is supported on you system. The results differ depending on lib, cpu and driver version. vainfo is also packaged in many linux distributions. Output on a kabylake microarchitecture based CPU will e.g. look like this:

libva info: VA-API version 1.5.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_4
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Broxton - 2.3.0
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple :	VAEntrypointVLD
VAProfileMPEG2Main :	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
VAProfileH264Main :	VAEntrypointVLD
VAProfileH264Main :	VAEntrypointEncSlice
VAProfileH264Main :	VAEntrypointEncSliceLP
VAProfileH264High :	VAEntrypointVLD
VAProfileH264High :	VAEntrypointEncSlice
VAProfileH264High :	VAEntrypointEncSliceLP
VAProfileH264MultiviewHigh :	VAEntrypointVLD
VAProfileH264MultiviewHigh :	VAEntrypointEncSlice
VAProfileH264StereoHigh :	VAEntrypointVLD
VAProfileH264StereoHigh :	VAEntrypointEncSlice
VAProfileVC1Simple :	VAEntrypointVLD
VAProfileVC1Main :	VAEntrypointVLD
VAProfileVC1Advanced :	VAEntrypointVLD
VAProfileNone :	VAEntrypointVideoProc
VAProfileJPEGBaseline :	VAEntrypointVLD
VAProfileJPEGBaseline :	VAEntrypointEncPicture
VAProfileVP8Version0_3 :	VAEntrypointVLD
VAProfileVP8Version0_3 :	VAEntrypointEncSlice
VAProfileHEVCMain :	VAEntrypointVLD
VAProfileHEVCMain :	VAEntrypointEncSlice
VAProfileHEVCMain10 :	VAEntrypointVLD
VAProfileVP9Profile0 :	VAEntrypointVLD

== VDPAU

https://en.wikipedia.org/wiki/VDPAU

== OMX / OpenMAX

https://en.wikipedia.org/wiki/OpenMAX

= Hardware Hardware en/decoder can be found in a number of chips. The probably most common type are GPUs or GPU components of CPUs and SOCs. Besides that there are also some dedicated de/encoder cards available. You probably can also use FPGAs but we didn't look into that (yet)

== Intel Intel implement the video acceleration in the GPU parts of their CPUs. Its mostly available in Desktop and Tablet CPUs but also in some Xeon Server CPUs. The hardware feature is called QuickSync an can be used via different APIs / Libs. Intel makes it not easy to user to figure out which CPU/GPU can do what, below are some links that might help.

https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video

https://en.wikipedia.org/wiki/Xeon

https://en.wikichip.org/wiki/intel/microarchitectures/gen9#Hardware_Accelerated_Video

https://en.wikichip.org/wiki/intel/microarchitectures/gen9.5#Hardware_Accelerated_Video

The ffmpeg wiki has a clear list of the capabilities each Intel GPU generation offers:

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

The TL;DR on this is: If your CPU is at least a SandyBridge based CPU with a GPU core to en/decode h264. If you need VP9 you should at least have an kabylake core. Depending of what exactly you do you will be able to have more than 10 streams in parallel.

To see the load of your GPU, which also will tell you how much headroom you have for more streams, you can use intels

intel_gpu_top

which is packaged e.g. on debian with 'intel-gpu-tools'.

== Nvidia As usual NVIDIA provides the worst user experience possible by limiting the feature not based on hardware capabilities but on the price of the card. This means you will not be able to en- / decode much stream in parallel if you don't pay a an absurd amount of money. But never the less you might already own a card …. Also NVIDIA basically only supports H264 and H265 on the encoder side. Depending on your card and driver you can use your card with VAAPI or need to go for the nvenc / nvdec

https://developer.nvidia.com/video-encode-decode-gpu-support-matrix

Number of Streams (h264 1080p30 8bit 4:2:0) / NVenc
Architecture	Highest Quality	Low Latency	Highest Quantity
Kepler	2	7
Maxwell 1st Gen	4	12
Maxwell 2nd Gen	6	13
Pascal/Volta	9	13	21
Turing	11	17

http://web.archive.org/web/20190505115654im_/https://developer.nvidia.com/sites/default/files/akamai/designworks/images-videocodec/VCSDK_TU003a.png

http://web.archive.org/web/20180913160347im_/https://developer.nvidia.com/sites/default/files/akamai/designworks/images-videocodec/nvenc_perf_B_001.png

http://web.archive.org/web/20170214143849im_/https://developer.nvidia.com/sites/default/files/akamai/designworks/images/VidEncode_HP_002.png

To see the load of your GPU use nvidia's

nvidia-smi pmon

== AMD / ATI === Radeon The AMD stand alone GPUs provide also video en/ decoder capabilties. With recent Linux Kernels e.g. the NAVI Gpus (Radeon GT 5xxx) can be used without binary drivers. The first AMD/ATI GPU with video accellerator was R600 architecture from 2007. AMD developed different accelerator suites over time.

E.g. a Radion gt 5500 XT provides the following encoders through libva

libva info: VA-API version 1.6.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.6 (libva 2.6.0)
vainfo: Driver version: Mesa Gallium driver 20.0.2 for AMD NAVI14 (DRM 3.35.0, 5.4.0-18-generic, LLVM 9.0.1)
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple :	VAEntrypointVLD
VAProfileMPEG2Main :	VAEntrypointVLD
VAProfileVC1Simple :	VAEntrypointVLD
VAProfileVC1Main :	VAEntrypointVLD
VAProfileVC1Advanced :	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
VAProfileH264Main :	VAEntrypointVLD
VAProfileH264Main :	VAEntrypointEncSlice
VAProfileH264High :	VAEntrypointVLD
VAProfileH264High :	VAEntrypointEncSlice
VAProfileHEVCMain :	VAEntrypointVLD
VAProfileHEVCMain :	VAEntrypointEncSlice
VAProfileHEVCMain10 :	VAEntrypointVLD
VAProfileJPEGBaseline :	VAEntrypointVLD
VAProfileVP9Profile0 :	VAEntrypointVLD
VAProfileVP9Profile2 :	VAEntrypointVLD
VAProfileNone :	VAEntrypointVideoProc

== Broadcom === Christal HD Broadcoms Christal HD cards where mostly used for decoding video on devices not capably to to it in CPU. It can only do decoding so it is not that relevant here.

https://en.wikipedia.org/wiki/Broadcom_Crystal_HD

=== Raspberry Pi 1-4 The SOCs used in the raspberrypis can be used for de/encoding of video. It can be accessed with OMX api which is present in gstreamer and ffmpeg

https://wiki.matthiasbock.net/index.php/Hardware-accelerated_video_playback_on_the_Raspberry_Pi

On an raspberrypi (1-4) the following omx features are supported (at least according to gstreamer)

omxmpeg2videodec: OpenMAX MPEG2 Video Decoder
omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
omxh263dec: OpenMAX H.263 Video Decoder
omxh264dec: OpenMAX H.264 Video Decoder
omxtheoradec: OpenMAX Theora Video Decoder
omxvp8dec: OpenMAX VP8 Video Decoder
omxmjpegdec: OpenMAX MJPEG Video Decoder
omxvc1dec: OpenMAX WMV Video Decoder
omxh264enc: OpenMAX H.264 Video Encoder
omxanalogaudiosink: OpenMAX Analog Audio Sink
omxhdmiaudiosink: OpenMAX HDMI Audio Sink

= Software

== FFMPEG

https://trac.ffmpeg.org/wiki/HWAccelIntro

https://trac.ffmpeg.org/wiki/Hardware/VAAPI,

== Gstreamer To see which de/encoder gstreamer can use on your system you can run e.g.:

sudo gst-inspect-1.0 vaapi

gstreamer1.0-vaapi needs to be installed for this to work. Replace vaapi with the API you want to use e.g. omx. The output will look like:

sudo gst-inspect-1.0 vaapi
Plugin Details:
Name vaapi
Description VA-API based elements
Filename /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so
Version 1.16.1
License LGPL
Source module gstreamer-vaapi
Source release date 2019-09-23
Binary package gstreamer-vaapi
Origin URL http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer
vaapih264enc: VA-API H264 encoder
vaapijpegenc: VA-API JPEG encoder
vaapivp8enc: VA-API VP8 encoder
vaapih265enc: VA-API H265 encoder
vaapisink: VA-API sink
vaapidecodebin: VA-API Decode Bin
vaapipostproc: VA-API video postprocessing
vaapih265dec: VA-API H265 decoder
vaapivp9dec: VA-API VP9 decoder
vaapivp8dec: VA-API VP8 decoder
vaapivc1dec: VA-API VC1 decoder
vaapih264dec: VA-API H264 decoder
vaapimpeg2dec: VA-API MPEG2 decoder
vaapijpegdec: VA-API JPEG decoder

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/ch01.html

https://gstreamer.freedesktop.org/documentation/tutorials/playback/hardware-accelerated-video-decoding.html