Differences

This shows you the differences between two versions of the page.

--- cdn [2025/02/06 01:04] – ischluff
+++ cdn [2025/05/22 11:22] (current) – ischluff
@@ Line 2: / Line 2: @@
 <bootnote>
-This documentation describes our Livestreaming CDN as of 2024.
+This documentation describes our Livestreaming CDN as of 2025.
 The File-CDN for media.ccc.de is described on [[software:voctoweb]].
@@ Line 13: / Line 13: @@
 == Architecture
-The CDN-Cascade has 5 stages: Master-Encoder, Ingest, Transcoder, Master-Relays and finally Edge-Relays.
+The CDN-Cascade has 6 stages: Master-Encoder, Ingest, Transcoder, Master-Relays, Edge-Relays and finally Load-balancers.
 {{:cdn_overview.png?1200|}}
@@ Line 43: / Line 43: @@
 # consul kv get /stream/s96/transcoder
 myloc-transcoder3
+# To reallocate a transcoder for a stuck stream
+consul kv delete /stream/s96/transcoder
 ```
@@ Line 59: / Line 62: @@
 The edge relays run nginx with caching proxy config and upstream set to the master relay.
-== Stream-URLs
+=== 6. Loadbalancer
-The resulting stream URLs are as follows:
-HLS (H.264 multi-quality + multi-language)
-* http://cdn.c3voc.de/hls/sX/native_hd.m3u8
-MPEG-DASH (VPx multi-quality + multi-language)
-* http://cdn.c3voc.de/dash/sX/manifest.mpd
-Moar HLS
-* http://cdn.c3voc.de/hls/sX/translated_hd.m3u8 (also multi-language, just a different default language)
-* http://cdn.c3voc.de/hls/sX/translated-2_hd.m3u8
-* http://cdn.c3voc.de/hls/sX/native_sd.m3u8 (only SD, but still multi-language)
-* http://cdn.c3voc.de/hls/sX/translated_sd.m3u8
-* http://cdn.c3voc.de/hls/sX/translated-2_sd.m3u8
-Audio-MP3:
-* http://cdn.c3voc.de/sX_native.mp3
-* http://cdn.c3voc.de/sX_translated.mp3
-* http://cdn.c3voc.de/sX_translated-2.mp3
-Audio-Opus:
-* http://cdn.c3voc.de/sX_native.opus
-* http://cdn.c3voc.de/sX_translated.opus
-* http://cdn.c3voc.de/sX_translated-2.opus
-== Loadbalancers
 The end-user facing domains ''streaming.media.ccc.de'' as well as ''cdn.c3voc.de'' are served by the load-balancers using DNS round robin.
-=== streaming.media.ccc.de
+==== streaming.media.ccc.de
 The load-balancers serve the [[:software:streamingwebsite|Streaming-Webseite]] under this domain. The sites run completely independent and are just kept in sync by the deployment script. With this procedure one or more load-balancers can easily be taken out of the DNS rotation if necessary.
-=== cdn.c3voc.de
+==== cdn.c3voc.de
 Requests to ''cdn.c3voc.de'' are handled by haproxy on the load-balancers with redirects to one of the edge relays. The redirects are performed following a preconfigured weight distribution.
@@ Line 99: / Line 75: @@
 ```
 # Get the current weight distribution, if not explicitly set the default is 100
-consul kv get -recurse /services/haproxy/backends/                                                                                                                                                                                                                                                         2025-02-06 0:54:27
+consul kv get -recurse /services/haproxy/backends/
 services/haproxy/backends/live.event.c3voc.de/weight:1
 services/haproxy/backends/relive.c3voc.de/weight:1
@@ Line 106: / Line 82: @@
 Because of the behavior of HLS/DASH clients if a client fetches a playlist file and gets a redirect, all further requests go to the same origin. With this trick the clients stay persistently on one relay until they are redirected.
-=== Dynamic SRT/RTMP endpoints with RTMP Auth
+== Stream-URLs
+The resulting stream URLs are as follows (replace @slug with your actual stream slug):
+HLS (H.264 multi-quality + multi-language)
+* http://cdn.c3voc.de/hls/@slug/native_hd.m3u8
+Moar HLS
+  * http://cdn.c3voc.de/hls/@slug/translated_hd.m3u8 - also multi-language, just a different default language
+  * http://cdn.c3voc.de/hls/@slug/translated-2_hd.m3u8
+  * http://cdn.c3voc.de/hls/@slug/native_sd.m3u8 - only SD, but still multi-language
+  * http://cdn.c3voc.de/hls/@slug/translated_sd.m3u8
+  * http://cdn.c3voc.de/hls/@slug/translated-2_sd.m3u8
+**Currently not active:**
+MPEG-DASH (VPx multi-quality + multi-language)
+* http://cdn.c3voc.de/dash/@slug/manifest.mpd
+Audio-MP3:
+* http://cdn.c3voc.de/@slug_native.mp3
+* http://cdn.c3voc.de/@slug_translated.mp3
+* http://cdn.c3voc.de/@slug_translated-2.mp3
+Audio-Opus:
+* http://cdn.c3voc.de/@slug_native.opus
+* http://cdn.c3voc.de/@slug_translated.opus
+* http://cdn.c3voc.de/@slug_translated-2.opus
+== Dynamic SRT/RTMP endpoints with RTMP Auth
 SRT and RTMP Endpoints can be created under https://ingest.c3voc.de/backend/ (Password in keepass under ansible/stream-api/htpasswd). After creating the endpoint the stream can be pushed to any ingest machine.
@@ Line 115: / Line 119: @@
 == Future improvements
-The diagram below shows the planned architecture as of 2020. In the meantime many of the improvements have been integrated into the CDN.
+The diagram below shows the envisioned architecture as of 2020. In the meantime many of the improvements have been integrated into the CDN.
-The main remaining items are as follows:
+Some more potential improvements are as follows:
  * Dynamically adapting the streaming-website to the existence/lack of a stream
- * Build a live status dashboard which shows the current stream->ingest->transcoder pipelines
+ * Building a live status dashboard which shows the current stream->ingest->transcoder pipelines
  * To gracefully shutdown a relay it could be instructed via consul KV to redirect all clients back to a load balancer while taking it out of the haproxy redirects. That way all clients would be redistributed to different relays.
+ * Build master relay redundancy using dynamic upload client and relay upstream configuration. (Relay upstreams need to be sticky, because the HLS playlists generated on the master relay are not deterministic)
+ * Use DNS rotation on ingest
+ * Finalize srtrelay meshing to allow pushed streams to be pulled from any ingest machine.
-=== Overview
 {{drawio>diagram1.png}}