This is an old revision of the document!
Video-Pis
Research-documentation for H264-encoding and streaming with Raspberry Pis.
gstreamer with Auvidea B101 HDMI-CSI2-Bridge
- max. supported resolution: 1080p30. Theoretically CSI-2 with two lanes (card-sized RPis; CM4-module has 4 lanes!) should be capable of handling 1080p50 in UYVY colorspace, but this fails so far, even as UYVY seems to be in use. (It's unclear if everything will literally melt, too ;)
- The iframe-peroid of 2 frames is quiet aggressive.
- I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!
Hint: For better reading pipelines are described as pipeline files which one can use with cat pipeline-file.txt | xargs gst-launch-1.0.
SMPTE2022-1 FEC streaming (multicast/unicast), audio and display-loop-out included
Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: blog-post by Mathieu.
Documentation: rtpst2022-1-fecdec, rtpst2022-1-fecenc
- Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
- Works per multicast as well as per unicast.
- Latency: ca. 1 second
- Receiver needs a static route towards 224.0.0.0/4 on eth0 in order to receive multicast.
- Receiver needs to be started before (first) launch of sender. Help needed! Issue at Mathieu.
- Don't forget to remove “netsim drop-probability=0.01” before flight!
Sender, using audio from HDMI-In and sending video-input to first HDMI-display port:
rtpbin
name=rtp
fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";'
v4l2src
device=/dev/video0
io-mode=0
do-timestamp=true
! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"
! tee
name=input
! v4l2h264enc
output-io-mode=0
extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;"
! "video/x-h264,profile=high, level=(string)4"
! h264parse
! mpegtsmux
name=mux
! rtpmp2tpay
ssrc=0
! rtp.send_rtp_sink_0
rtp.send_rtp_src_0
! udpsink
host=224.0.23.1
port=5000
multicast-iface=eth0
rtp.send_fec_src_0_0
! udpsink
host=224.0.23.1
port=5002
async=false
multicast-iface=eth0
rtp.send_fec_src_0_1
! udpsink
host=224.0.23.1
port=5004
async=false
multicast-iface=eth0
alsasrc
device=hw:tc358743
do-timestamp=true
! "audio/x-raw,format=S16LE,rate=48000,channels=2"
! audioconvert
! voaacenc
bitrate=48000
! aacparse
! queue
! mux.
input.
! queue
! videoconvert
! fbdevsink
device="/dev/fb0"
sync=false
Receiver, with audio-out on Headphone-jack:
rtpbin
latency=100
fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";'
name=rtp
udpsrc
address=224.0.23.1
port=5002
caps="application/x-rtp, payload=96"
! queue
! rtp.recv_fec_sink_0_0
udpsrc
address=224.0.23.1
port=5004
caps="application/x-rtp, payload=96"
! queue
! rtp.recv_fec_sink_0_1
udpsrc
address=224.0.23.1
port=5000
caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33"
! queue
! netsim
drop-probability=0.01
! rtp.recv_rtp_sink_0
rtp.
! queue
! decodebin
name=decode
! videoconvert
! queue
! kmssink
decode.
! audioconvert
! queue
! alsasink
card-name="bcm2835 Headphones"
SRT streaming (unicast), audio and display-loop-out included
- Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
- Latency: ca. 0.7 seconds
- SRT-desired latency is with 40ms VERY aggressive! The suggested value is 2.5 times the round-trip-time (RTT) between both hosts.
- starting/restarting-order seems strange and needs to be investigated.
Sender, using audio from HDMI-In and sending video-input to display:
v4l2src
device=/dev/video0
io-mode=0
do-timestamp=true
! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"
! tee
name=input
! v4l2h264enc
output-io-mode=0
extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;"
! "video/x-h264,profile=high, level=(string)4"
! mpegtsmux
name=mux
! srtsink
uri=srt://:8888
latency=40
mode=listener
wait-for-connection=true
input.
! queue
! videoconvert
! fbdevsink
sync=false
alsasrc
device=hw:tc358743
do-timestamp=true
! "audio/x-raw,format=S16LE,rate=48000,channels=2"
! audioconvert
! voaacenc
bitrate=48000
! aacparse
! queue
! mux.
Receiver, with audio-out on Headphone-jack:
srtclientsrc
latency=20
mode=caller
uri=srt://${ipAddress}:8888
! tsdemux
latency=40
name=demux
demux.
! queue
! decodebin
name=decode
! videoconvert
! queue
! kmssink
demux.
! queue
! avdec_aac
! audioconvert
! alsasink
card-name="bcm2835 Headphones"
RIST streaming (multicast/unicast), audio and display-loop-out included
RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too?
- Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
- Latency: reducable to 0.5s
- most errors
Sender, using audio from HDMI-In and sending video-input to display:
v4l2src
device=/dev/video0
io-mode=0
do-timestamp=true
! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"
! tee
name=input
! v4l2h264enc
output-io-mode=0
extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2,sequence_header_mode=0;"
! "video/x-h264,profile=high, level=(string)4"
! mpegtsmux
name=mux
! rtpmp2tpay
! ristsink
address=224.0.23.1
port=5004
multicast-iface=eth0
input.
! queue
! videoconvert
! fbdevsink
sync=false \
alsasrc
device=hw:tc358743
do-timestamp=true
! "audio/x-raw,format=S16LE,rate=48000,channels=2"
! audioconvert
! voaacenc
bitrate=48000
! aacparse
! queue
! mux.
Receiver, with audio-out on headphone-jack:
ristsrc
address=224.0.23.1
receiver-buffer=200
reorder-section=20
! rtpmp2tdepay
! tsdemux
latency=50
name=demux
demux.
! queue
! decodebin
name=decode
! videoconvert
! queue
! kmssink
demux.
! queue
! avdec_aac
! audioconvert
! alsasink
card-name="bcm2835 Headphones"