Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
hardware:video-pis [2024/03/20 00:39] – adding SRT streaming with Auvidea B101 HDMI-CSI2-Bridge, audio and display-loop-out included wtf | hardware:video-pis [2024/04/03 21:13] (current) – wtf | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | = Video-Pis | ||
+ | |||
Research-documentation for H264-encoding and streaming with Raspberry Pis. | Research-documentation for H264-encoding and streaming with Raspberry Pis. | ||
- | = gstreamer | + | == gstreamer |
- | == SMPTE2022-1 FEC per UDP streaming | + | * max. supported resolution: 1080p30. [[https:// |
+ | * The iframe-peroid of 2 frames is quiet aggressive. | ||
+ | * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate! | ||
- | Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer: [[https:// | + | Hint: For better reading pipelines are described as pipeline files which one can use with `cat pipeline-file.txt | xargs gst-launch-1.0`. |
+ | |||
+ | === SMPTE2022-1 FEC streaming (multicast/ | ||
+ | |||
+ | Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: [[https:// | ||
Documentation: | Documentation: | ||
Line 12: | Line 20: | ||
* Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024. | * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024. | ||
* Works per multicast as well as per unicast. | * Works per multicast as well as per unicast. | ||
- | * Latency: | + | * Latency: |
* Receiver needs a static route towards 224.0.0.0/4 on eth0 in order to receive multicast. | * Receiver needs a static route towards 224.0.0.0/4 on eth0 in order to receive multicast. | ||
* Receiver needs to be started before (first) launch of sender. Help needed! [[https:// | * Receiver needs to be started before (first) launch of sender. Help needed! [[https:// | ||
- | * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate! | ||
- | * The iframe-peroid of 2 frames is quiet aggressive. | ||
* Don't forget to remove " | * Don't forget to remove " | ||
- | Sender, using Audio from HDMI-In and sending video-input to display: | + | Sender, using audio from HDMI-In and sending video-input to first HDMI-display |
- | gst-launch-1.0 | ||
- | rtpbin name=rtp fec-encoders=' | ||
- | v4l2src device=/ | ||
- | tee name=input ! \ | ||
- | v4l2h264enc output-io-mode=0 extra-controls=" | ||
- | " | ||
- | mpegtsmux name=mux ! rtpmp2tpay ssrc=0 ! rtp.send_rtp_sink_0 | ||
- | rtp.send_rtp_src_0 ! udpsink host=${ipAddress} port=5000 multicast-iface=eth0 \ | ||
- | rtp.send_fec_src_0_0 ! udpsink host=${ipAddress} port=5002 async=false multicast-iface=eth0 \ | ||
- | rtp.send_fec_src_0_1 ! udpsink host=${ipAddress} port=5004 async=false multicast-iface=eth0 \ | ||
- | alsasrc device=hw: | ||
- | audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux. \ | ||
- | input. ! queue ! videoconvert ! fbdevsink device="/ | ||
+ | ``` | ||
+ | rtpbin | ||
+ | name=rtp | ||
+ | fec-encoders=' | ||
+ | |||
+ | v4l2src | ||
+ | device=/ | ||
+ | io-mode=0 | ||
+ | do-timestamp=true | ||
+ | ! " | ||
+ | ! tee | ||
+ | name=input | ||
+ | ! v4l2h264enc | ||
+ | output-io-mode=0 | ||
+ | extra-controls=" | ||
+ | ! " | ||
+ | ! h264parse | ||
+ | ! mpegtsmux | ||
+ | name=mux | ||
+ | ! rtpmp2tpay | ||
+ | ssrc=0 | ||
+ | ! rtp.send_rtp_sink_0 | ||
+ | |||
+ | rtp.send_rtp_src_0 | ||
+ | ! udpsink | ||
+ | host=224.0.23.1 | ||
+ | port=5000 | ||
+ | multicast-iface=eth0 | ||
+ | |||
+ | rtp.send_fec_src_0_0 | ||
+ | ! udpsink | ||
+ | host=224.0.23.1 | ||
+ | port=5002 | ||
+ | async=false | ||
+ | multicast-iface=eth0 | ||
+ | |||
+ | rtp.send_fec_src_0_1 | ||
+ | ! udpsink | ||
+ | host=224.0.23.1 | ||
+ | port=5004 | ||
+ | async=false | ||
+ | multicast-iface=eth0 | ||
+ | |||
+ | alsasrc | ||
+ | device=hw: | ||
+ | do-timestamp=true | ||
+ | ! " | ||
+ | ! audioconvert | ||
+ | ! voaacenc | ||
+ | bitrate=48000 | ||
+ | ! aacparse | ||
+ | ! queue | ||
+ | ! mux. | ||
+ | |||
+ | input. | ||
+ | ! queue | ||
+ | ! videoconvert | ||
+ | ! fbdevsink | ||
+ | device="/ | ||
+ | sync=false | ||
+ | ``` | ||
Receiver, with audio-out on Headphone-jack: | Receiver, with audio-out on Headphone-jack: | ||
- | gst-launch-1.0 \ | + | ``` |
- | rtpbin latency=100 fec-decoders=' | + | rtpbin |
- | udpsrc address=224.0.23.1 port=5002 caps=" | + | |
- | | + | |
- | | + | |
- | queue ! netsim drop-probability=0.01 | + | |
- | rtp. ! queue ! decodebin name=decode | + | udpsrc |
- | decode. ! audioconvert ! queue ! alsasink card-name=" | + | |
+ | | ||
+ | | ||
+ | ! queue | ||
+ | ! rtp.recv_fec_sink_0_0 | ||
+ | udpsrc | ||
+ | address=224.0.23.1 | ||
+ | port=5004 | ||
+ | caps=" | ||
+ | ! queue | ||
+ | ! rtp.recv_fec_sink_0_1 | ||
- | == SRT streaming with Auvidea B101 HDMI-CSI2-Bridge, audio and display-loop-out included | + | udpsrc |
+ | address=224.0.23.1 | ||
+ | port=5000 | ||
+ | caps=" | ||
+ | ! queue | ||
+ | ! netsim | ||
+ | drop-probability=0.01 | ||
+ | ! rtp.recv_rtp_sink_0 | ||
+ | |||
+ | rtp. | ||
+ | ! queue | ||
+ | ! decodebin | ||
+ | name=decode | ||
+ | ! videoconvert | ||
+ | ! queue | ||
+ | ! kmssink | ||
+ | |||
+ | decode. | ||
+ | ! audioconvert | ||
+ | ! queue | ||
+ | ! alsasink | ||
+ | card-name=" | ||
+ | ``` | ||
+ | |||
+ | === SRT streaming (unicast), audio and display-loop-out included | ||
* Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024. | * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024. | ||
- | * Latency: | + | * Latency: |
+ | * SRT-desired latency is with 40ms VERY aggressive! The suggested value is 2.5 times the round-trip-time (RTT) between both hosts. | ||
* starting/ | * starting/ | ||
- | Sender: | + | Sender, using audio from HDMI-In and sending video-input to display: |
- | gst-launch-1.0 -v \ | + | ``` |
- | | + | v4l2src |
- | | + | device=/ |
- | | + | |
- | | + | |
- | | + | ! " |
- | input. ! queue ! videoconvert ! fbdevsink sync=false | + | ! tee |
- | alsasrc device=hw: | + | name=input |
+ | ! v4l2h264enc | ||
+ | output-io-mode=0 | ||
+ | | ||
+ | ! " | ||
+ | ! mpegtsmux | ||
+ | name=mux | ||
+ | ! srtsink | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | input. | ||
+ | ! queue | ||
+ | ! videoconvert | ||
+ | ! fbdevsink | ||
+ | | ||
+ | |||
+ | alsasrc | ||
+ | | ||
+ | | ||
+ | ! "audio/ | ||
+ | ! audioconvert | ||
+ | ! voaacenc | ||
+ | | ||
+ | ! aacparse | ||
+ | ! queue | ||
+ | ! mux. | ||
+ | ``` | ||
+ | |||
+ | Receiver, with audio-out on Headphone-jack: | ||
+ | |||
+ | ``` | ||
+ | srtclientsrc | ||
+ | latency=20 | ||
+ | mode=caller | ||
+ | uri=srt:// | ||
+ | ! tsdemux | ||
+ | latency=40 | ||
+ | name=demux | ||
+ | |||
+ | demux. | ||
+ | ! queue | ||
+ | ! decodebin | ||
+ | name=decode | ||
+ | ! videoconvert | ||
+ | ! queue | ||
+ | ! kmssink | ||
+ | |||
+ | demux. | ||
+ | ! queue | ||
+ | ! avdec_aac | ||
+ | ! audioconvert | ||
+ | ! alsasink | ||
+ | card-name=" | ||
+ | ``` | ||
+ | |||
+ | === RIST streaming (multicast/ | ||
+ | |||
+ | RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too? | ||
+ | |||
+ | * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024. | ||
+ | * Latency: reducable to 0.5s | ||
+ | * most errors | ||
+ | |||
+ | Sender, using audio from HDMI-In and sending video-input to display: | ||
+ | |||
+ | ``` | ||
+ | v4l2src | ||
+ | device=/ | ||
+ | io-mode=0 | ||
+ | do-timestamp=true | ||
+ | ! " | ||
+ | ! tee | ||
+ | name=input | ||
+ | ! v4l2h264enc | ||
+ | output-io-mode=0 | ||
+ | extra-controls=" | ||
+ | ! " | ||
+ | ! mpegtsmux | ||
+ | name=mux | ||
+ | ! rtpmp2tpay | ||
+ | ! ristsink | ||
+ | address=224.0.23.1 | ||
+ | port=5004 | ||
+ | multicast-iface=eth0 | ||
+ | |||
+ | input. | ||
+ | ! queue | ||
+ | ! videoconvert | ||
+ | ! fbdevsink | ||
+ | sync=false \ | ||
| | ||
- | Receiver: | + | alsasrc |
+ | device=hw: | ||
+ | do-timestamp=true | ||
+ | ! " | ||
+ | ! audioconvert | ||
+ | ! voaacenc | ||
+ | bitrate=48000 | ||
+ | ! aacparse | ||
+ | ! queue | ||
+ | ! mux. | ||
+ | ``` | ||
+ | |||
+ | Receiver, with audio-out on headphone-jack: | ||
+ | |||
+ | ``` | ||
+ | ristsrc | ||
+ | address=224.0.23.1 | ||
+ | receiver-buffer=200 | ||
+ | reorder-section=20 | ||
+ | ! rtpmp2tdepay | ||
+ | ! tsdemux | ||
+ | latency=50 | ||
+ | name=demux | ||
- | gst-launch-1.0 \ | + | demux. |
- | | + | ! queue |
- | | + | ! decodebin |
- | demux. ! queue ! avdec_aac ! audioconvert ! alsasink card-name=" | + | |
+ | ! videoconvert | ||
+ | ! queue | ||
+ | ! kmssink | ||
+ | demux. | ||
+ | ! queue | ||
+ | ! avdec_aac | ||
+ | ! audioconvert | ||
+ | ! alsasink | ||
+ | card-name=" | ||
+ | ``` |