hardware:video-pis

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
hardware:video-pis [2024/03/21 20:23] – add RIST wtfhardware:video-pis [2026/08/13 08:46] (current) – [Video-Pis] clarifcation of current RPi4-scope wtf
Line 1: Line 1:
-= Video-Pis+# Video-Pis
  
-Research-documentation for H264-encoding and streaming with Raspberry Pis.+Research-documentation for H264-encoding and streaming with Raspberry Pis. Document created when RPi 4 was new, so these examples incorporate hardware-encoding and need adaption for RPi 5.
  
-== gstreamer with Auvidea B101 HDMI-CSI2-Bridge+## gstreamer with Auvidea B101 HDMI-CSI2-Bridge
  
- * max. supported resolution: 1080p30. [[https://forums.raspberrypi.com/viewtopic.php?t=268787|Theoretically]] CSI-2 with two lanes should be capable of handling 1080p50 in UYVY colorspace, but this fails so far, even as UYVY seems to be in use. (It's unclear if everything will literally melt, too ;)+ * max. supported resolution: 1080p30. [[https://forums.raspberrypi.com/viewtopic.php?t=268787|Theoretically]] CSI-2 with two lanes (card-sized RPis; CM4-module has 4 lanes!) should be capable of handling 1080p50 in UYVY colorspace, but this fails so far, even as UYVY seems to be in use. (It's unclear if everything will literally melt, too ;)
  * The iframe-peroid of 2 frames is quiet aggressive.  * The iframe-peroid of 2 frames is quiet aggressive.
  * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!  * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!
  
-=== SMPTE2022-1 FEC streaming (multicast/unicast), audio and display-loop-out included+Hint: For better reading pipelines are described as pipeline files which one can use with `cat pipeline-file.txt | xargs gst-launch-1.0`. 
 + 
 +### SMPTE2022-1 FEC streaming (multicast/unicast), audio and display-loop-out included
  
 Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: [[https://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html|blog-post by Mathieu]].  Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: [[https://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html|blog-post by Mathieu]]. 
Line 23: Line 25:
  * Don't forget to remove "netsim drop-probability=0.01" before flight!  * Don't forget to remove "netsim drop-probability=0.01" before flight!
  
-Sender, using Audio from HDMI-In and sending video-input to display: +Sender, using audio from HDMI-In and sending video-input to first HDMI-display port:
- +
-  gst-launch-1.0  \ +
-      rtpbin name=rtp fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";' \ +
-        v4l2src device=/dev/video0 io-mode=0 do-timestamp=true ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" ! \ +
-        tee name=input ! \ +
-        v4l2h264enc output-io-mode=0 extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" ! \ +
-        "video/x-h264,profile=high, level=(string)4" ! h264parse ! \ +
-        mpegtsmux name=mux ! rtpmp2tpay ssrc=0 ! rtp.send_rtp_sink_0  \ +
-          rtp.send_rtp_src_0 ! udpsink host=${ipAddress} port=5000 multicast-iface=eth0 \ +
-          rtp.send_fec_src_0_0 ! udpsink host=${ipAddress} port=5002 async=false multicast-iface=eth0 \ +
-          rtp.send_fec_src_0_1 ! udpsink host=${ipAddress} port=5004 async=false multicast-iface=eth0 \ +
-        alsasrc device=hw:tc358743 do-timestamp=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! \ +
-        audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux. \ +
-        input. ! queue ! videoconvert ! fbdevsink device="/dev/fb0" sync=false  +
  
 +<code>
 +  rtpbin 
 +    name=rtp 
 +    fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";'
 +  
 +  v4l2src 
 +    device=/dev/video0 
 +    io-mode=0
 +    do-timestamp=true 
 +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"
 +  ! tee 
 +    name=input
 +  ! v4l2h264enc
 +    output-io-mode=0 
 +    extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" 
 +  ! "video/x-h264,profile=high, level=(string)4" 
 +  ! h264parse 
 +  ! mpegtsmux 
 +    name=mux 
 +  ! rtpmp2tpay 
 +    ssrc=0 
 +  ! rtp.send_rtp_sink_0
 +  
 +  rtp.send_rtp_src_0 
 +  ! udpsink 
 +    host=224.0.23.1 
 +    port=5000 
 +    multicast-iface=eth0 
 +  
 +  rtp.send_fec_src_0_0 
 +  ! udpsink 
 +    host=224.0.23.1 
 +    port=5002 
 +    async=false 
 +    multicast-iface=eth0
 +  
 +  rtp.send_fec_src_0_1 
 +  ! udpsink 
 +    host=224.0.23.1 
 +    port=5004 
 +    async=false 
 +    multicast-iface=eth0
 +  
 +  alsasrc
 +    device=hw:tc358743 
 +    do-timestamp=true 
 +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2" 
 +  ! audioconvert 
 +  ! voaacenc 
 +    bitrate=48000 
 +  ! aacparse 
 +  ! queue 
 +  ! mux.
 +  
 +  input. 
 +  ! queue 
 +  ! videoconvert 
 +  ! fbdevsink 
 +    device="/dev/fb0" 
 +    sync=false
 +</code>
  
 Receiver, with audio-out on Headphone-jack: Receiver, with audio-out on Headphone-jack:
  
-  gst-launch-1.0 \ +<code> 
-    rtpbin latency=100 fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";' name=rtp \ +  rtpbin  
-    udpsrc address=224.0.23.1 port=5002 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_0 \ +    latency=100  
-    udpsrc address=224.0.23.1 port=5004 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_1 \ +    fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";'  
-    udpsrc address=224.0.23.1 port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33" ! \ +    name=rtp 
-      queue ! netsim drop-probability=0.01 ! rtp.recv_rtp_sink_0 \ +       
-    rtp. ! queue ! decodebin name=decode  ! videoconvert ! queue !  kmssink \ +  udpsrc  
-    decode. ! audioconvert ! queue ! alsasink card-name="bcm2835 Headphones"+    address=224.0.23.1  
 +    port=5002  
 +    caps="application/x-rtp, payload=96"  
 +  ! queue  
 +  ! rtp.recv_fec_sink_0_0 
 +   
 +  udpsrc 
 +    address=224.0.23.1  
 +    port=5004  
 +    caps="application/x-rtp, payload=96"  
 +  ! queue  
 +  ! rtp.recv_fec_sink_0_1 
 +   
 +  udpsrc 
 +    address=224.0.23.1 
 +    port=5000 
 +    caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33"  
 +  ! queue  
 +  ! netsim  
 +      drop-probability=0.01  
 +  ! rtp.recv_rtp_sink_0 
 +   
 +  rtp.  
 +  ! queue  
 +  ! decodebin  
 +      name=decode 
 +  ! videoconvert  
 +  ! queue  
 +  ! kmssink 
 +   
 +  decode.  
 +  ! audioconvert  
 +  ! queue  
 +  ! alsasink  
 +      card-name="bcm2835 Headphones" 
 +</code>
  
- +### SRT streaming (unicast), audio and display-loop-out included
-=== SRT streaming (unicast), audio and display-loop-out included+
  
  * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.  * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
Line 61: Line 144:
 Sender, using audio from HDMI-In and sending video-input to display: Sender, using audio from HDMI-In and sending video-input to display:
  
-  gst-launch-1.0 -v \ +<code> 
-    v4l2src device=/dev/video0 io-mode=0 do-timestamp=true ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" ! \ +  v4l2src  
-    tee name=input ! \ +      device=/dev/video0  
-    v4l2h264enc output-io-mode=0 extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" ! \ +      io-mode=0  
-    "video/x-h264,profile=high, level=(string)4" ! \ +      do-timestamp=true  
-    mpegtsmux name=mux ! srtsink uri=srt://:8888 latency=40 mode=listener wait-for-connection=true \ +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  
-    input. ! queue ! videoconvert ! fbdevsink sync=false \ +  ! tee  
-    alsasrc device=hw:tc358743 do-timestamp=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux.  +      name=input 
-    +  ! v4l2h264enc  
 +      output-io-mode=0  
 +      extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" 
 +  ! "video/x-h264,profile=high, level=(string)4" 
 +  ! mpegtsmux  
 +      name=mux  
 +  ! srtsink  
 +      uri=srt://:8888  
 +      latency=40  
 +      mode=listener  
 +      wait-for-connection=true 
 +   
 +  input.  
 +  ! queue  
 +  ! videoconvert  
 +  ! fbdevsink  
 +      sync=false 
 +   
 +  alsasrc  
 +      device=hw:tc358743  
 +      do-timestamp=true  
 +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2"  
 +  ! audioconvert  
 +  ! voaacenc  
 +      bitrate=48000  
 +  ! aacparse  
 +  ! queue  
 +  ! mux.  
 +</code> 
 Receiver, with audio-out on Headphone-jack: Receiver, with audio-out on Headphone-jack:
  
-  gst-launch-1.0 \ +<code> 
-    srtclientsrc latency=20 mode=caller uri=srt://${ipAddress}:8888 ! \ +  srtclientsrc  
-    tsdemux latency=40 name=demux demux. ! queue ! decodebin name=decode ! videoconvert ! queue ! kmssink \ +      latency=20  
-    demux. ! queue ! avdec_aac ! audioconvert ! alsasink card-name="bcm2835 Headphones"+      mode=caller  
 +      uri=srt://${ipAddress}:8888  
 +  ! tsdemux  
 +      latency=40  
 +      name=demux  
 +   
 +  demux.  
 +  ! queue  
 +  ! decodebin  
 +      name=decode  
 +  ! videoconvert  
 +  ! queue  
 +  ! kmssink 
 +   
 +  demux.  
 +  ! queue  
 +  ! avdec_aac  
 +  ! audioconvert  
 +  ! alsasink  
 +      card-name="bcm2835 Headphones" 
 +</code>
  
- +### RIST streaming (multicast/unicast), audio and display-loop-out included
-=== RIST streaming (multicast/unicast), audio and display-loop-out included+
  
 RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too? RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too?
Line 88: Line 219:
 Sender, using audio from HDMI-In and sending video-input to display: Sender, using audio from HDMI-In and sending video-input to display:
  
-  gst-launch-1.0  \ +<code> 
-    v4l2src device=/dev/video0 io-mode=0 do-timestamp=true ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" ! \ +  v4l2src  
-    tee name=input ! \ +      device=/dev/video0  
-    v4l2h264enc output-io-mode=0 extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2,sequence_header_mode=0;" ! \ +      io-mode=0  
-    "video/x-h264,profile=high, level=(string)4" ! \ +      do-timestamp=true  
-    mpegtsmux name=mux ! rtpmp2tpay ! ristsink address=224.0.23.1 port=5004  sender-buffer=100  multicast-iface=eth0 \ +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  
-    input. ! queue ! videoconvert ! fbdevsink sync=false \ +  ! tee  
-    alsasrc device=hw:tc358743 do-timestamp=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux.+      name=input  
 +  ! v4l2h264enc  
 +      output-io-mode=0  
 +      extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2,sequence_header_mode=0;"  
 +  ! "video/x-h264,profile=high, level=(string)4"  
 +  ! mpegtsmux  
 +      name=mux  
 +  ! rtpmp2tpay  
 +  ! ristsink  
 +      address=224.0.23.1  
 +      port=5004  
 +      multicast-iface=eth0 
 +   
 +  input.  
 +  ! queue  
 +  ! videoconvert  
 +  ! fbdevsink  
 +      sync=false \ 
 +       
 +  alsasrc  
 +      device=hw:tc358743  
 +      do-timestamp=true  
 +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2"  
 +  ! audioconvert  
 +  ! voaacenc  
 +      bitrate=48000  
 +  ! aacparse  
 +  ! queue  
 +  ! mux. 
 +</code>
  
 Receiver, with audio-out on headphone-jack: Receiver, with audio-out on headphone-jack:
  
-gst-launch-1.0  \ +<code> 
-    ristsrc address=224.0.23.1 receiver-buffer=100 reorder-section=5 ! rtpmp2tdepay ! \ +  ristsrc  
-    tsdemux latency=50 name=demux demux. ! queue ! decodebin name=decode ! videoconvert ! queue ! kmssink \ +      address=224.0.23.1  
-    demux. ! queue ! avdec_aac ! audioconvert ! alsasink card-name="bcm2835 Headphones" +      receiver-buffer=200  
 +      reorder-section=20  
 +  ! rtpmp2tdepay  
 +  ! tsdemux  
 +      latency=50  
 +      name=demux  
 +   
 +  demux.  
 +  ! queue  
 +  ! decodebin  
 +      name=decode  
 +  ! videoconvert  
 +  ! queue  
 +  ! kmssink 
 +   
 +  demux.  
 +  ! queue  
 +  ! avdec_aac  
 +  ! audioconvert  
 +  ! alsasink  
 +      card-name="bcm2835 Headphones" 
 +</code>
  • hardware/video-pis.1711052624.txt.gz
  • Last modified: 2024/03/21 20:23
  • by wtf