c3tracker:setup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
c3tracker:setup [2025/09/19 18:34] – andic3tracker:setup [2026/08/19 14:36] (current) – commanderred
Line 1: Line 1:
-== Project setup+## Project setup
  
   * Create tracker project (you can also clone a matching project to have prefilled properties)   * Create tracker project (you can also clone a matching project to have prefilled properties)
Line 8: Line 8:
  
  
-==== optimal properties from current project ====+#### optimal properties from current project
  
 Replace `camp2023` with your chosen tracker project slug. Replace `camp2023` with your chosen tracker project slug.
  
-  Meta.Acronym                    camp2023 +``` 
-  Meta.Album                      Chaos Communication Camp 2023 +Meta.Acronym                    camp2023 
-  Meta.License                    Licensed to the public under http://creativecommons.org/licenses/by/4.0 +Meta.Album                      Chaos Communication Camp 2023 
-  Meta.Year                       2023+Meta.License                    Licensed to the public under http://creativecommons.org/licenses/by/4.0 
 +Meta.Year                       2023
      
-  Processing.Auphonic.Enable      no +Processing.Auphonic.Enable      no 
-  Processing.BasePath             /video/ +Processing.BasePath             /video/ 
-  Processing.MasterMe.Enable      yes +Processing.MasterMe.Enable      yes 
-  Processing.Path.Intros          /video/intros/camp2023 +Processing.Path.Intros          /video/intros/camp2023 
-  Processing.Path.Outro           /video/intros/camp2023/outro.ts+Processing.Path.Outro           /video/intros/camp2023/outro.ts
      
-  Publishing.Upload.SkipSlaves    speedy,tweety,blade1,blade2,blade3,blade4 +Publishing.UploadTarget         releasing.c3voc.de:/video/encoded/camp2023/ 
-  Publishing.UploadTarget         releasing.c3voc.de:/video/encoded/camp2023/ +Publishing.Tags                 <additional tags> 
-  Publishing.Tags                 <additional tags> +Publishing.Voctoweb.Enable      yes 
-  Publishing.Voctoweb.Enable      yes +Publishing.Voctoweb.Path        /cdn.media.ccc.de/events/camp2023 
-  Publishing.Voctoweb.Path        /cdn.media.ccc.de/events/camp2023 +Publishing.Voctoweb.Slug        camp2023 
-  Publishing.Voctoweb.Slug        camp2023 +Publishing.Voctoweb.Thumbpath   /static.media.ccc.de/conferences/camp2023 
-  Publishing.Voctoweb.Thumbpath   /static.media.ccc.de/conferences/camp2023 +Publishing.YouTube.Category     27 
-  Publishing.YouTube.Category     27 +Publishing.YouTube.Enable       yes 
-  Publishing.YouTube.Enable       yes +Publishing.YouTube.Playlists    <meep> 
-  Publishing.YouTube.Playlists    <meep> +Publishing.YouTube.Privacy      <one of: public, unlisted, private> 
-  Publishing.YouTube.Privacy      <one of: public, unlisted, private> +Publishing.YouTube.Token        <meep>
-  Publishing.YouTube.Token        <meep>+
      
-  Record.Container                TS +Record.Container                TS 
-  Record.EndPadding               300 +Record.EndPadding               300 
-  Record.Slides                   yes +Record.Slides                   yes 
-  Record.StartPadding             300  +Record.StartPadding             300  
- +```
- +
-=== Worker Filter Examples+
  
 +### Worker Filter Examples
  
 +  
 +  # get all non-master tickets
   EncodingProfile.IsMaster=no   EncodingProfile.IsMaster=no
 +  
 +  # get all master tickets
   EncodingProfile.IsMaster=yes   EncodingProfile.IsMaster=yes
 +  
 +  # get all recording tickets
   EncodingProfile.IsMaster=   EncodingProfile.IsMaster=
 +  
 +  # get all tickets in this room.
   Fahrplan.Room=Servus.at Lab   Fahrplan.Room=Servus.at Lab
  
  
-Please note that the conditions in the "project to worker group" filter are currently always evaluated with logical OR. +Please note that the conditions in the "project to worker group" filter are currently always evaluated with logical OR.
  
 Specifying a property with an empty value, which is often done for `EncodingProfile.IsMaster`, will match if this property does not exist at all on a ticket. So for `EncodingProfile.IsMaster`, specifying an empty filter will match on recording tickets which never have this property. Specifying a property with an empty value, which is often done for `EncodingProfile.IsMaster`, will match if this property does not exist at all on a ticket. So for `EncodingProfile.IsMaster`, specifying an empty filter will match on recording tickets which never have this property.
-== Pipeline setup during event+ 
 + 
 +{{.:pasted:20250919-215601.png}} 
 +## Pipeline setup during event
  
 During event setup of the pipeline, you have to decide if you want leave the MPEG TS snippets only on the [[hardware:encoder|recording cubes]] or also rsync them to a central storage: During event setup of the pipeline, you have to decide if you want leave the MPEG TS snippets only on the [[hardware:encoder|recording cubes]] or also rsync them to a central storage:
  
  
-=== Simple: single-room setup (Variant A)+### Simple: single-room setup (Variant A)
  
 {{drawio>c3tracker:setup-simple.png}} {{drawio>c3tracker:setup-simple.png}}
Line 81: Line 91:
 <panel type="danger" title="Attention">Since tracker filters are joined via OR and not AND, this setup cannot be extended to multiple rooms without hacks if you want to e.g. limit on-site encoding to master formats. Use the  `CRS_ROOM` filter in bundlewrap if you need both tracker filters and room-specific encoding workers.</panel> <panel type="danger" title="Attention">Since tracker filters are joined via OR and not AND, this setup cannot be extended to multiple rooms without hacks if you want to e.g. limit on-site encoding to master formats. Use the  `CRS_ROOM` filter in bundlewrap if you need both tracker filters and room-specific encoding workers.</panel>
  
-=== centralised storage (rsync) (Variant B)+### centralised storage (rsync) (Variant B)
  
 {{drawio>c3tracker:setup-central-storage.png}} {{drawio>c3tracker:setup-central-storage.png}}
  
  
-This variant is typically used for events with more than one room. For bigger events we use the dedicated [[hardware:event-storage|storage]] server in the event server rack, for smaller events a USB connected hard drive to one of the minions might be sufficient. Each recording encoder exposes the files via rsyncd, which are pulled by an rsync process running on the storage machine.+This variant is typically used for events with more than one room. 
  
-For each encoderX start rsync on the central storage: `sudo systemctl start rsync-from-encoder@encoderX.lan.c3voc.de`+For bigger events we use the dedicated [[hardware:event-storage|storage]] server in the event server rack, for smaller events one of the minions is typically sufficient. If you use a minion as storage machine, you might want to configure `10.73.0.2` (aka `storage.lan.c3voc.de`) as secondary IP address to make things easier:
  
-Then, start tracker workers on storage: `sudo systemctl start crs-worker.target` (only needed if you don't use `storage.lan.c3voc.de` - worker scripts get started automatically)+  sudo ip a add dev br0 10.73.200.24/16
  
  
-==== Minion setup+Each recording encoder exposes the files via rsyncd, which are pulled by an rsync process running on this storage machine.
  
-To allow the encoding workers to do their job, they need to mount the storage first: `sudo crs-mount <storage location>` 
  
-After mounting, you can start the tracker encoding workers: `sudo systemctl start crs-encoding.service`+For each encoderX start rsync on the storage machine: 
  
-The minion VMs running inside our event colo case automatically mount `storage.lan.c3voc.de` via cifs and start their worker scripts. You usually do not need to touch them.+  sudo systemctl start rsync-from-encoder@encoderX.lan.c3voc.de 
 + 
 +If you use a minion as storage machine, you have start the tracker manually (on storage.lan.c3voc.de they are started by default):  
 + 
 +  sudo systemctl start crs-worker.target
      
 +which starts
  
-==== Encoder PCs as release encoding workers+- crs-recording-scheduler (A) 
 +- crs-mount4cut.service   (B) 
 +- crs-cut-postprocessor   (C) 
 +- crs-encoding0, crs-encoding1 (D) 
 +- crs-postencoding        (E) 
 +- crs-postprocessing      (F) 
 + 
 + 
 + 
 +#### Minion setup 
 + 
 +To allow get additional release encoding workers on the (other) minions, you need to connect to these an mount the storage machine first:  
 + 
 +  ssh minionX.lan.c3voc.de 
 +  sudo crs-mount <storage location> 
 + 
 +After mounting, you can start the tracker encoding workers:  
 + 
 +  sudo systemctl start crs-encoding0.service 
 + 
 +The virtual minion's living in [[hardware:server-case|event colo case]] automatically mount `storage.lan.c3voc.de` via CIFS and start their worker scripts. You usually do not need to touch them. 
 +   
 + 
 +#### Encoder PCs as release encoding workers
  
 At small events, when all talks are finished for today, you can use the recording encoder PCs to encode master MP4 files.  At small events, when all talks are finished for today, you can use the recording encoder PCs to encode master MP4 files. 
  
-First: Stop voctocore:+First – stop voctocore:
  
   sudo systemctl stop voctomix2-voctocore   sudo systemctl stop voctomix2-voctocore
Line 121: Line 158:
      
  
-=== decentralised pipeline aka "even more samba" (Variant 3)+### decentralised pipeline aka "even more samba" (Variant C)
  
-<panel type="danger" title="Attention">The "decentralized pipeline (Variant 3)" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust/improve the documentation here.</panel>+<panel type="danger" title="Attention">The "decentralized pipeline" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust/improve the documentation here.</panel>
  
-Similar to variant 2, but extended to work with multiple rooms. Instead of using rsync, recorded snippets remain on the encoding cubes and ''/video/fuse/$event/$room'' are exposed via samba to the minions, while the encoded and tmp files live on one "central" minion; all other minions mount ''/video/encoded'' and ''/video/tmp'' from the primary minion [reasoning: the tracker cannot guarantee that the machine which encoded a talk also does the postprocessing (upload) step, so all minions have to see the same files].+Similar to variant B, but extended to work with multiple rooms. Instead of using rsync, recorded snippets remain on the encoding cubes and ''/video/fuse/$event/$room'' are exposed via samba to the minions, while the encoded and tmp files live on one "central" minion; all other minions mount ''/video/encoded'' and ''/video/tmp'' from the primary minion [reasoning: the tracker cannot guarantee that the machine which encoded a talk also does the postprocessing (upload) step, so all minions have to see the same files].
  
 Tracker filters have to be set only for the recording cubes, minions do not require any filters (but on smaller events without many minions, a ''EncodingProfile.IsMaster=yes'' filter can be a good idea, so sub formats won't crowd out the queues — they can always be encoded off-site later). Tracker filters have to be set only for the recording cubes, minions do not require any filters (but on smaller events without many minions, a ''EncodingProfile.IsMaster=yes'' filter can be a good idea, so sub formats won't crowd out the queues — they can always be encoded off-site later).
Line 153: Line 190:
  
  
-==== Old example with systemd units and case 1 and 5, which was used during jev22 in Munich:+#### Old example with systemd units and case 1 and 5, which was used during jev22 in Munich:
  
 {{drawio>c3tracker:setup-variant-3.png}} {{drawio>c3tracker:setup-variant-3.png}}
  
 optional: configure `10.73.0.2` (aka `storage.lan.c3voc.de`) on the master minion as secondary ip optional: configure `10.73.0.2` (aka `storage.lan.c3voc.de`) on the master minion as secondary ip
 +
 +  sudo ip a add dev br0 10.73.200.24/16
  
 on recording cubes, mount or copy the intros from their source – here `storage.lan.c3voc.de` on recording cubes, mount or copy the intros from their source – here `storage.lan.c3voc.de`
Line 200: Line 239:
        
  
-==== Old example with custom screenrc and case 5 and 6:+#### Old example with custom screenrc and case 5 and 6:
  
 on recording cube, without intros either copy or mount the intros from their source   on recording cube, without intros either copy or mount the intros from their source  
  • c3tracker/setup.1758306886.txt.gz
  • Last modified: 2025/09/19 18:34
  • by andi