hardware:video-pis

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
hardware:video-pis [2024/04/03 19:11] – [gstreamer with Auvidea B101 HDMI-CSI2-Bridge] wtfhardware:video-pis [2026/08/13 08:46] (current) – [Video-Pis] clarifcation of current RPi4-scope wtf
Line 1: Line 1:
-= Video-Pis+# Video-Pis
  
-Research-documentation for H264-encoding and streaming with Raspberry Pis.+Research-documentation for H264-encoding and streaming with Raspberry Pis. Document created when RPi 4 was new, so these examples incorporate hardware-encoding and need adaption for RPi 5.
  
-== gstreamer with Auvidea B101 HDMI-CSI2-Bridge+## gstreamer with Auvidea B101 HDMI-CSI2-Bridge
  
  * max. supported resolution: 1080p30. [[https://forums.raspberrypi.com/viewtopic.php?t=268787|Theoretically]] CSI-2 with two lanes (card-sized RPis; CM4-module has 4 lanes!) should be capable of handling 1080p50 in UYVY colorspace, but this fails so far, even as UYVY seems to be in use. (It's unclear if everything will literally melt, too ;)  * max. supported resolution: 1080p30. [[https://forums.raspberrypi.com/viewtopic.php?t=268787|Theoretically]] CSI-2 with two lanes (card-sized RPis; CM4-module has 4 lanes!) should be capable of handling 1080p50 in UYVY colorspace, but this fails so far, even as UYVY seems to be in use. (It's unclear if everything will literally melt, too ;)
Line 9: Line 9:
  * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!  * I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!
  
-=== SMPTE2022-1 FEC streaming (multicast/unicast), audio and display-loop-out included+Hint: For better reading pipelines are described as pipeline files which one can use with `cat pipeline-file.txt | xargs gst-launch-1.0`. 
 + 
 +### SMPTE2022-1 FEC streaming (multicast/unicast), audio and display-loop-out included
  
 Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: [[https://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html|blog-post by Mathieu]].  Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer, done in MPEGTS via UDP with 2D-FEC: [[https://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html|blog-post by Mathieu]]. 
Line 23: Line 25:
  * Don't forget to remove "netsim drop-probability=0.01" before flight!  * Don't forget to remove "netsim drop-probability=0.01" before flight!
  
-Sender, using Audio from HDMI-In and sending video-input to display:+Sender, using audio from HDMI-In and sending video-input to first HDMI-display port:
  
-Hint: For better reading pipelines are described as pipeline files which one can use with `gst-launch-1.0 -f pipeline-file.txt`. +<code> 
- +  rtpbin  
-``` +    name=rtp  
-rtpbin  +    fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";' 
-  name=rtp  +   
-  fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";' +  v4l2src  
- +    device=/dev/video0  
-v4l2src  +    io-mode=0 
-  device=/dev/video0  +    do-timestamp=true  
-  io-mode=0 +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" 
-  do-timestamp=true  +  ! tee  
-! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" +    name=input 
-! tee  +  ! v4l2h264enc 
-  name=input +    output-io-mode=0  
-! v4l2h264enc +    extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;"  
-  output-io-mode=0  +  ! "video/x-h264,profile=high, level=(string)4"  
-  extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;"  +  ! h264parse  
-! "video/x-h264,profile=high, level=(string)4"  +  ! mpegtsmux  
-! h264parse  +    name=mux  
-! mpegtsmux  +  ! rtpmp2tpay  
-  name=mux  +    ssrc=0  
-! rtpmp2tpay  +  ! rtp.send_rtp_sink_0 
-  ssrc=0  +   
-! rtp.send_rtp_sink_0 +  rtp.send_rtp_src_0  
- +  ! udpsink  
-rtp.send_rtp_src_0  +    host=224.0.23.1  
-! udpsink  +    port=5000  
-  host=224.0.23.1  +    multicast-iface=eth0  
-  port=5000  +   
-  multicast-iface=eth0  +  rtp.send_fec_src_0_0  
- +  ! udpsink  
-rtp.send_fec_src_0_0  +    host=224.0.23.1  
-! udpsink  +    port=5002  
-  host=224.0.23.1  +    async=false  
-  port=5002  +    multicast-iface=eth0 
-  async=false  +   
-  multicast-iface=eth0 +  rtp.send_fec_src_0_1  
- +  ! udpsink  
-rtp.send_fec_src_0_1  +    host=224.0.23.1  
-! udpsink  +    port=5004  
-  host=224.0.23.1  +    async=false  
-  port=5004  +    multicast-iface=eth0 
-  async=false  +   
-  multicast-iface=eth0 +  alsasrc 
- +    device=hw:tc358743  
-alsasrc +    do-timestamp=true  
-  device=hw:tc358743  +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2"  
-  do-timestamp=true  +  ! audioconvert  
-! "audio/x-raw,format=S16LE,rate=48000,channels=2"  +  ! voaacenc  
-! audioconvert  +    bitrate=48000  
-! voaacenc  +  ! aacparse  
-  bitrate=48000  +  ! queue  
-! aacparse  +  ! mux. 
-! queue  +   
-! mux. +  input.  
- +  ! queue  
-input.  +  ! videoconvert  
-! queue  +  ! fbdevsink  
-! videoconvert  +    device="/dev/fb0"  
-! fbdevsink  +    sync=false 
-  device="/dev/fb0"  +</code>
-  sync=false +
-```+
  
 Receiver, with audio-out on Headphone-jack: Receiver, with audio-out on Headphone-jack:
  
-``` +<code> 
-rtpbin  +  rtpbin  
-  latency=100  +    latency=100  
-  fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";'  +    fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";'  
-  name=rtp +    name=rtp 
-     +       
-udpsrc  +  udpsrc  
-  address=224.0.23.1  +    address=224.0.23.1  
-  port=5002  +    port=5002  
-  caps="application/x-rtp, payload=96"  +    caps="application/x-rtp, payload=96"  
-! queue  +  ! queue  
-! rtp.recv_fec_sink_0_0 +  ! rtp.recv_fec_sink_0_0 
- +   
-udpsrc +  udpsrc 
-  address=224.0.23.1  +    address=224.0.23.1  
-  port=5004  +    port=5004  
-  caps="application/x-rtp, payload=96"  +    caps="application/x-rtp, payload=96"  
-! queue  +  ! queue  
-! rtp.recv_fec_sink_0_1 +  ! rtp.recv_fec_sink_0_1 
- +   
-udpsrc +  udpsrc 
-  address=224.0.23.1 +    address=224.0.23.1 
-  port=5000 +    port=5000 
-  caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33"  +    caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33"  
-! queue  +  ! queue  
-! netsim  +  ! netsim  
-    drop-probability=0.01  +      drop-probability=0.01  
-! rtp.recv_rtp_sink_0 +  ! rtp.recv_rtp_sink_0 
- +   
-rtp.  +  rtp.  
-! queue  +  ! queue  
-! decodebin  +  ! decodebin  
-    name=decode +      name=decode 
-! videoconvert  +  ! videoconvert  
-! queue  +  ! queue  
-! kmssink +  ! kmssink 
- +   
-decode.  +  decode.  
-! audioconvert  +  ! audioconvert  
-! queue  +  ! queue  
-! alsasink  +  ! alsasink  
-    card-name="bcm2835 Headphones" +      card-name="bcm2835 Headphones" 
-```+</code>
  
-=== SRT streaming (unicast), audio and display-loop-out included+### SRT streaming (unicast), audio and display-loop-out included
  
  * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.  * Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
Line 144: Line 144:
 Sender, using audio from HDMI-In and sending video-input to display: Sender, using audio from HDMI-In and sending video-input to display:
  
-``` +<code> 
-v4l2src  +  v4l2src  
-    device=/dev/video0  +      device=/dev/video0  
-    io-mode=0  +      io-mode=0  
-    do-timestamp=true  +      do-timestamp=true  
-! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  
-! tee  +  ! tee  
-    name=input +      name=input 
-! v4l2h264enc  +  ! v4l2h264enc  
-    output-io-mode=0  +      output-io-mode=0  
-    extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" +      extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" 
-! "video/x-h264,profile=high, level=(string)4" +  ! "video/x-h264,profile=high, level=(string)4" 
-! mpegtsmux  +  ! mpegtsmux  
-    name=mux  +      name=mux  
-! srtsink  +  ! srtsink  
-    uri=srt://:8888  +      uri=srt://:8888  
-    latency=40  +      latency=40  
-    mode=listener  +      mode=listener  
-    wait-for-connection=true +      wait-for-connection=true 
- +   
-input.  +  input.  
-! queue  +  ! queue  
-! videoconvert  +  ! videoconvert  
-! fbdevsink  +  ! fbdevsink  
-    sync=false +      sync=false 
- +   
-alsasrc  +  alsasrc  
-    device=hw:tc358743  +      device=hw:tc358743  
-    do-timestamp=true  +      do-timestamp=true  
-! "audio/x-raw,format=S16LE,rate=48000,channels=2"  +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2"  
-! audioconvert  +  ! audioconvert  
-! voaacenc  +  ! voaacenc  
-    bitrate=48000  +      bitrate=48000  
-! aacparse  +  ! aacparse  
-! queue  +  ! queue  
-! mux.  +  ! mux.  
-```+</code>
  
 Receiver, with audio-out on Headphone-jack: Receiver, with audio-out on Headphone-jack:
  
-``` +<code> 
-srtclientsrc  +  srtclientsrc  
-    latency=20  +      latency=20  
-    mode=caller  +      mode=caller  
-    uri=srt://${ipAddress}:8888  +      uri=srt://${ipAddress}:8888  
-! tsdemux  +  ! tsdemux  
-    latency=40  +      latency=40  
-    name=demux  +      name=demux  
- +   
-demux.  +  demux.  
-! queue  +  ! queue  
-! decodebin  +  ! decodebin  
-    name=decode  +      name=decode  
-! videoconvert  +  ! videoconvert  
-! queue  +  ! queue  
-! kmssink +  ! kmssink 
- +   
-demux.  +  demux.  
-! queue  +  ! queue  
-! avdec_aac  +  ! avdec_aac  
-! audioconvert  +  ! audioconvert  
-! alsasink  +  ! alsasink  
-    card-name="bcm2835 Headphones" +      card-name="bcm2835 Headphones" 
-```+</code>
  
-=== RIST streaming (multicast/unicast), audio and display-loop-out included+### RIST streaming (multicast/unicast), audio and display-loop-out included
  
 RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too? RIST seems to be the new kid around the corner of SRT. No FEC, but re-transmission-requests with RTCP, which shall work via multicast, too?
Line 219: Line 219:
 Sender, using audio from HDMI-In and sending video-input to display: Sender, using audio from HDMI-In and sending video-input to display:
  
-``` +<code> 
-v4l2src  +  v4l2src  
-    device=/dev/video0  +      device=/dev/video0  
-    io-mode=0  +      io-mode=0  
-    do-timestamp=true  +      do-timestamp=true  
-! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  +  ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601"  
-! tee  +  ! tee  
-    name=input  +      name=input  
-! v4l2h264enc  +  ! v4l2h264enc  
-    output-io-mode=0  +      output-io-mode=0  
-    extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2,sequence_header_mode=0;"  +      extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2,sequence_header_mode=0;"  
-! "video/x-h264,profile=high, level=(string)4"  +  ! "video/x-h264,profile=high, level=(string)4"  
-! mpegtsmux  +  ! mpegtsmux  
-    name=mux  +      name=mux  
-! rtpmp2tpay  +  ! rtpmp2tpay  
-! ristsink  +  ! ristsink  
-    address=224.0.23.1  +      address=224.0.23.1  
-    port=5004  +      port=5004  
-    multicast-iface=eth0 +      multicast-iface=eth0 
- +   
-input.  +  input.  
-! queue  +  ! queue  
-! videoconvert  +  ! videoconvert  
-! fbdevsink  +  ! fbdevsink  
-    sync=false \ +      sync=false \ 
-     +       
-alsasrc  +  alsasrc  
-    device=hw:tc358743  +      device=hw:tc358743  
-    do-timestamp=true  +      do-timestamp=true  
-! "audio/x-raw,format=S16LE,rate=48000,channels=2"  +  ! "audio/x-raw,format=S16LE,rate=48000,channels=2"  
-! audioconvert  +  ! audioconvert  
-! voaacenc  +  ! voaacenc  
-    bitrate=48000  +      bitrate=48000  
-! aacparse  +  ! aacparse  
-! queue  +  ! queue  
-! mux. +  ! mux. 
-```+</code>
  
 Receiver, with audio-out on headphone-jack: Receiver, with audio-out on headphone-jack:
  
-``` +<code> 
-ristsrc  +  ristsrc  
-    address=224.0.23.1  +      address=224.0.23.1  
-    receiver-buffer=200  +      receiver-buffer=200  
-    reorder-section=20  +      reorder-section=20  
-! rtpmp2tdepay  +  ! rtpmp2tdepay  
-! tsdemux  +  ! tsdemux  
-    latency=50  +      latency=50  
-    name=demux  +      name=demux  
- +   
-demux.  +  demux.  
-! queue  +  ! queue  
-! decodebin  +  ! decodebin  
-    name=decode  +      name=decode  
-! videoconvert  +  ! videoconvert  
-! queue  +  ! queue  
-! kmssink +  ! kmssink 
- +   
-demux.  +  demux.  
-! queue  +  ! queue  
-! avdec_aac  +  ! avdec_aac  
-! audioconvert  +  ! audioconvert  
-! alsasink  +  ! alsasink  
-    card-name="bcm2835 Headphones" +      card-name="bcm2835 Headphones" 
-```+</code>
  • hardware/video-pis.1712171490.txt.gz
  • Last modified: 2024/04/03 19:11
  • by wtf