gstreamer

Since 2020 there is an implementation of SMPTE 2022-1 in gstreamer: blog-post by Mathieu.

Documentation: rtpst2022-1-fecdec, rtpst2022-1-fecenc

• Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
• Works per multicast as well as per unicast.
• Latency: 1-1.1 seconds
• Receiver needs a static route towards 224.0.0.0/4 on eth0 in order to receive multicast.
• Receiver needs to be started before (first) launch of sender. Help needed! Issue at Mathieu.
• I/O-mode = auto. Result unclear, it seems popular stating DMAbuf here. Investigate!
• The iframe-peroid of 2 frames is quiet aggressive.
• Don't forget to remove “netsim drop-probability=0.01” before flight!

Sender, using Audio from HDMI-In and sending video-input to display:

gst-launch-1.0  \
    rtpbin name=rtp fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=4\ columns\=4";' \
      v4l2src device=/dev/video0 io-mode=0 do-timestamp=true ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" ! \
      tee name=input ! \
      v4l2h264enc output-io-mode=0 extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" ! \
      "video/x-h264,profile=high, level=(string)4" ! h264parse ! \
      mpegtsmux name=mux ! rtpmp2tpay ssrc=0 ! rtp.send_rtp_sink_0  \
        rtp.send_rtp_src_0 ! udpsink host=${ipAddress} port=5000 multicast-iface=eth0 \
        rtp.send_fec_src_0_0 ! udpsink host=${ipAddress} port=5002 async=false multicast-iface=eth0 \
        rtp.send_fec_src_0_1 ! udpsink host=${ipAddress} port=5004 async=false multicast-iface=eth0 \
      alsasrc device=hw:tc358743 do-timestamp=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! \
      audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux. \
      input. ! queue ! videoconvert ! fbdevsink device="/dev/fb0" sync=false  

Receiver, with audio-out on Headphone-jack:

gst-launch-1.0 \
  rtpbin latency=100 fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=100000000";' name=rtp \
  udpsrc address=224.0.23.1 port=5002 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_0 \
  udpsrc address=224.0.23.1 port=5004 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_1 \
  udpsrc address=224.0.23.1 port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33" ! \
    queue ! netsim drop-probability=0.01 ! rtp.recv_rtp_sink_0 \
  rtp. ! queue ! decodebin name=decode  ! videoconvert ! queue !  kmssink \
  decode. ! audioconvert ! queue ! alsasink card-name="bcm2835 Headphones"

• Tested by wtf with an Raspberry Pi 4B+ with Raspberry Pi OS 2023-10-10, Debian Bookworm each, in March 2024.
• Latency: 0.6-0.8 seconds
• starting/restarting-order seems strange and needs to be investigated.

Sender:

gst-launch-1.0 -v \
  v4l2src device=/dev/video0 io-mode=0 do-timestamp=true ! "video/x-raw,framerate=30/1,format=UYVY,colorimetry=bt601" ! \
  tee name=input ! \
  v4l2h264enc output-io-mode=0 extra-controls="controls,h264_profile=4,h264_level=11,video_bitrate=15000000,video_bitrate_mode=0,h264_i_frame_period=2;" ! \
  "video/x-h264,profile=high, level=(string)4" ! \
  mpegtsmux name=mux ! srtsink uri=srt://:8888 latency=40 mode=listener wait-for-connection=true \
  input. ! queue ! videoconvert ! fbdevsink sync=false \
  alsasrc device=hw:tc358743 do-timestamp=true ! audio/x-raw,format=S16LE,rate=48000,channels=2 ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux. 
  

Receiver:

gst-launch-1.0 \
  srtclientsrc latency=20 mode=caller uri=srt://${ipAddress}:8888 ! \
  tsdemux latency=20 name=demux demux. ! queue ! decodebin name=decode ! videoconvert ! queue ! kmssink \
  demux. ! queue ! avdec_aac ! audioconvert ! alsasink card-name="bcm2835 Headphones"