cdn

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
cdn [2025/11/03 11:23] – ischluffcdn [2026/07/26 22:27] (current) – external edit 127.0.0.1
Line 1: Line 1:
-= CDN+# CDN
  
 <bootnote> <bootnote>
Line 7: Line 7:
 </bootnote> </bootnote>
  
-==== Repo Links+#### Repo Links
   * Stream-api, Upload-Proxy, Upload-Server: https://github.com/voc/stream-api   * Stream-api, Upload-Proxy, Upload-Server: https://github.com/voc/stream-api
   * rtmp-auth daemon: https://github.com/voc/rtmp-auth   * rtmp-auth daemon: https://github.com/voc/rtmp-auth
   * Transcoding-Script: https://github.com/voc/transcoding   * Transcoding-Script: https://github.com/voc/transcoding
  
-== Architecture+## Architecture
 The CDN-Cascade has 6 stages: Encoder, Ingest, Transcoder, Origin-Relays, Edge-Relays and finally Load-balancers. The CDN-Cascade has 6 stages: Encoder, Ingest, Transcoder, Origin-Relays, Edge-Relays and finally Load-balancers.
 {{:cdn_overview.drawio.png|}} {{:cdn_overview.drawio.png|}}
  
-=== 1. Encoder+### 1. Encoder
 The initial stream encoding is created near the stage on the encoder PC running voctomix or any video-encoder for third party streams. The initial stream encoding is created near the stage on the encoder PC running voctomix or any video-encoder for third party streams.
 The encoding usually contains a 1080p25 H264 video signal, the slides as 1080p5 and up to 3 tracks of audio encoded in AAC (Native, Translated, Translated-2). The encoding usually contains a 1080p25 H264 video signal, the slides as 1080p5 and up to 3 tracks of audio encoded in AAC (Native, Translated, Translated-2).
Line 23: Line 23:
 The origin encoding is pushed as MPEG-TS via SRT to the ingest stage. The origin encoding is pushed as MPEG-TS via SRT to the ingest stage.
  
-=== 2. Ingest+### 2. Ingest
 The ingest machines run both [[https://github.com/voc/srtrelay|srtrelay]] and nginx-rtmp to receive pushed streams from the encoders. This also serves as the entry point for third party streams. The ingest machines run both [[https://github.com/voc/srtrelay|srtrelay]] and nginx-rtmp to receive pushed streams from the encoders. This also serves as the entry point for third party streams.
  
Line 33: Line 33:
 ``` ```
  
-=== 3. Transcoder+### 3. Transcoder
 The stream transcoding transforms the mpegts stream into multiple segmented streams (one per quality) via one big ffmpeg script. Additionally it creates thumbnails and preview images. The stream transcoding transforms the mpegts stream into multiple segmented streams (one per quality) via one big ffmpeg script. Additionally it creates thumbnails and preview images.
 The transcoding script can be found here https://github.com/voc/transcoding. The transcoding script can be found here https://github.com/voc/transcoding.
Line 52: Line 52:
 The transcoding outputs are directly pushed with HTTP uploads to the Origin relays. To ensure proper retries the uploads are sent over a local http-proxy (https://github.com/voc/stream-api/tree/master/cmd/upload-proxy). The transcoding outputs are directly pushed with HTTP uploads to the Origin relays. To ensure proper retries the uploads are sent over a local http-proxy (https://github.com/voc/stream-api/tree/master/cmd/upload-proxy).
  
-=== 4. Origin Relays+### 4. Origin Relays
 Origin relays run the upload server (https://github.com/voc/stream-api/tree/master/cmd/upload-server) which receives the http uploads and stores the files on local disk. Origin relays run the upload server (https://github.com/voc/stream-api/tree/master/cmd/upload-server) which receives the http uploads and stores the files on local disk.
 The upload server has some additional logic to rewrite the HLS playlists on the fly in order to properly handle stream restarts. The upload server has some additional logic to rewrite the HLS playlists on the fly in order to properly handle stream restarts.
Line 58: Line 58:
 The segmented streams, thumbnails etc. are currently served by nginx. The segmented streams, thumbnails etc. are currently served by nginx.
  
-=== 5. Edge-Relays+### 5. Edge-Relays
 The edge relays run nginx with caching proxy config and upstream set to the active origin relay. The edge relays run nginx with caching proxy config and upstream set to the active origin relay.
  
-=== 6. Loadbalancer+### 6. Loadbalancer
 The end-user facing domains ''streaming.media.ccc.de'' as well as ''cdn.c3voc.de'' are served by the load-balancers using DNS round robin. The end-user facing domains ''streaming.media.ccc.de'' as well as ''cdn.c3voc.de'' are served by the load-balancers using DNS round robin.
  
-==== streaming.media.ccc.de+#### streaming.media.ccc.de
 The load-balancers serve the [[:software:streamingwebsite|Streaming-Webseite]] under this domain. The sites run completely independent and are just kept in sync by the deployment script. With this procedure one or more load-balancers can easily be taken out of the DNS rotation if necessary. The load-balancers serve the [[:software:streamingwebsite|Streaming-Webseite]] under this domain. The sites run completely independent and are just kept in sync by the deployment script. With this procedure one or more load-balancers can easily be taken out of the DNS rotation if necessary.
  
-==== cdn.c3voc.de+#### cdn.c3voc.de
 Requests to ''cdn.c3voc.de'' are handled by haproxy on the load-balancers with redirects to one of the edge relays. The redirects are performed following a preconfigured weight distribution. Requests to ''cdn.c3voc.de'' are handled by haproxy on the load-balancers with redirects to one of the edge relays. The redirects are performed following a preconfigured weight distribution.
  
Line 81: Line 81:
 Because of the behavior of HLS/DASH clients if a client fetches a playlist file and gets a redirect, all further requests go to the same origin. With this trick the clients stay persistently on one relay until they are redirected. Because of the behavior of HLS/DASH clients if a client fetches a playlist file and gets a redirect, all further requests go to the same origin. With this trick the clients stay persistently on one relay until they are redirected.
  
-== Stream-URLs+## Stream-URLs
  
 The resulting stream URLs are as follows (replace @slug with your actual stream slug): The resulting stream URLs are as follows (replace @slug with your actual stream slug):
Line 109: Line 109:
 * http://cdn.c3voc.de/@slug_translated.opus * http://cdn.c3voc.de/@slug_translated.opus
 * http://cdn.c3voc.de/@slug_translated-2.opus * http://cdn.c3voc.de/@slug_translated-2.opus
-== Dynamic SRT/RTMP endpoints with RTMP Auth+## Dynamic SRT/RTMP endpoints with RTMP Auth
 SRT and RTMP Endpoints can be created under https://ingest.c3voc.de/backend/ (Password in keepass under ansible/stream-api/htpasswd). After creating the endpoint the stream can be pushed to any ingest machine. SRT and RTMP Endpoints can be created under https://ingest.c3voc.de/backend/ (Password in keepass under ansible/stream-api/htpasswd). After creating the endpoint the stream can be pushed to any ingest machine.
  
Line 116: Line 116:
 The SRT-URL would be : ''%%srt://ingest.c3voc.de:1337?streamid=publish/qtest23/<token>%%'' The SRT-URL would be : ''%%srt://ingest.c3voc.de:1337?streamid=publish/qtest23/<token>%%''
  
-== Future improvements+## Future improvements
  
 The diagram below shows the envisioned architecture as of 2020. In the meantime many of the improvements have been integrated into the CDN. The diagram below shows the envisioned architecture as of 2020. In the meantime many of the improvements have been integrated into the CDN.
  • cdn.1762169021.txt.gz
  • Last modified: 2025/11/03 11:23
  • by ischluff