c3tracker:setup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
c3tracker:setup [2022/12/28 20:31] – andic3tracker:setup [2023/09/02 09:08] (current) – kunsi
Line 7: Line 7:
   * Compare project properties with previous instalment of the same event   * Compare project properties with previous instalment of the same event
  
-optimal properties from current project+==== optimal properties from current project ====
  
 ``` ```
-Project +Meta.Acronym                    camp2023 
-Project.Slug eh19 +Meta.Album                      Chaos Communication Camp 2023 
-Processing +Meta.License                    Licensed to the public under http://creativecommons.org/licenses/by/4.0 
-Processing.Loudnorm.Enable yes +Meta.Year                       2023 
-Processing.BasePath /video + 
-Processing.Path.Intros /video/intros/eh19 +Processing.Auphonic.Enable      no 
-Processing.Path.Outro /video/intros/eh19/outro.ts +Processing.BasePath             /video/ 
-Publishing +Processing.MasterMe.Enable      yes 
-Publishing.Path /video/encoded/eh19 +Processing.Path.Intros          /video/intros/camp2023 
-Publishing.Upload.SkipSlaves speedy,tweety,blade1,blade2,blade3,blade4 +Processing.Path.Outro           /video/intros/camp2023/outro.ts 
-Publishing.UploadOptions -i /video/upload-key + 
-Publishing.UploadTarget upload@releasing.c3voc.de:/video/encoded/eh19 +Publishing.Path                 /video/encoded/camp2023/ 
-Publishing.Voctoweb.Enable yes +Publishing.Upload.SkipSlaves    speedy,tweety,blade1,blade2,blade3,blade4 
-Publishing.Voctoweb.Path /cdn.media.ccc.de/events/eh2019/ +Publishing.UploadTarget         upload@releasing.c3voc.de:/video/encoded/camp2023/ 
-Publishing.Voctoweb.Thumbpath /static.media.ccc.de/conferences/eh2019 +Publishing.Voctoweb.Enable      yes 
-Publishing.Voctoweb.Slug eh19 +Publishing.Voctoweb.Path        /cdn.media.ccc.de/events/camp2023 
-Publishing.Voctoweb.Tags easterhegg, Wien, c3w +Publishing.Voctoweb.Slug        camp2023 
-Publishing.Voctoweb.Url https://media.ccc.de/v/ +Publishing.Voctoweb.Tags  <additional tags> 
-Publishing.YouTube.Category 27 +Publishing.Voctoweb.Thumbpath   /static.media.ccc.de/conferences/camp2023 
-Publishing.YouTube.Enable yes +Publishing.YouTube.Category     27 
-Publishing.YouTube.Privacy public +Publishing.YouTube.Enable       yes 
-Publishing.YouTube.Tags easterhegg, Wien, bun intended, Chaos Computer Club Wien, c3w +Publishing.YouTube.Playlists    <meep> 
-Publishing.YouTube.TitlePrefix Easterhegg 2019 - +Publishing.YouTube.Privacy      <one of: public, unlisted, private> 
-Publishing.YouTube.Token 1/XXXXXXXXXXXX +Publishing.YouTube.Tags         <additional tags> 
-Publishing.Mastodon.Enable yes +Publishing.YouTube.Token        <meep> 
-Publishing.Twitter.Enable yes + 
-Record +Record.Container                TS 
-Record.Container TS +Record.EndPadding               300 
-Record.Slides yes+Record.Slides                   yes 
 +Record.StartPadding             300 
 ``` ```
  
Line 50: Line 51:
 ``` ```
  
 +Please note that the conditions in the "project to worker group" filter are currently evaluated with logical OR. 
  
 +Specifying a property with an empty value, which is often done for EncodingProfile.IsMaster, will match if this property does not exist at all on a ticket. So for EncodingProfile.IsMaster, specifying an empty filter will match on recording tickets which never have this property.
 == Pipeline setup during event == Pipeline setup during event
  
Line 57: Line 60:
  
 === Simple: decentralised classic (Variant 2) === Simple: decentralised classic (Variant 2)
 +
 +{{drawio>c3tracker:setup-simple.png}}
 +
  
 This variant is only practical if you have only one room, or at least one release encoder (aka [[hardware:Minion]]) for each recording cube.  This variant is only practical if you have only one room, or at least one release encoder (aka [[hardware:Minion]]) for each recording cube. 
Line 73: Line 79:
  
 === centralised storage (rsync) (Variant 1) === centralised storage (rsync) (Variant 1)
 +
 +{{drawio>c3tracker:setup-central-storage.png}}
 +
  
 The first variant is typically used for events with more than one room. For bigger events we use the dedicated [[hardware:event-storage|storage]] server in the event server rack, for smaller events a USB connected hard drive to one of the minions might be sufficient. Each recording cube exposes the files via rsyncd, which are pulled by an rsync process running inside a screen on the storage pc. The first variant is typically used for events with more than one room. For bigger events we use the dedicated [[hardware:event-storage|storage]] server in the event server rack, for smaller events a USB connected hard drive to one of the minions might be sufficient. Each recording cube exposes the files via rsyncd, which are pulled by an rsync process running inside a screen on the storage pc.
Line 78: Line 87:
 For each encoderX start rsync on the central storage: `sudo systemctl start rsync-from-encoder@encoderX.lan.c3voc.de` For each encoderX start rsync on the central storage: `sudo systemctl start rsync-from-encoder@encoderX.lan.c3voc.de`
  
-Then, start tracker workers on storage: `sudo systemctl start crs-worker.target`+Then, start tracker workers on storage: `sudo systemctl start crs-worker.target` (only needed if you don't use `storage.lan.c3voc.de` - worker scripts get started automatically)
  
 ==== Minion setup ==== Minion setup
Line 85: Line 94:
  
 After mounting, you can start the tracker encoding workers: `sudo systemctl start crs-encoding.service` After mounting, you can start the tracker encoding workers: `sudo systemctl start crs-encoding.service`
 +
 +The minion VMs running inside our event colo case automatically mount `storage.lan.c3voc.de` via cifs and start their worker scripts. You usually do not need to touch them.
      
 ==== Cube as worker setup ==== Cube as worker setup
Line 104: Line 115:
 === decentralised pipeline (Variant 3) === decentralised pipeline (Variant 3)
  
-<panel type="danger" title="Outdated">The "decentralized pipeline (Variant 3)" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust the documentation here.</panel>+<panel type="danger" title="Attention">The "decentralized pipeline (Variant 3)" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust/improve the documentation here.</panel>
  
 Similar to variant 2, but the release encoder (minion) only mounts the /video/fuse/$room/ from each recording cube. The encoded and tmp files life on one minion, the secondary minions mount /video/encoded and /video/tmp from the primary minion. [Reason: It is not guaranteed that the minion which encoded a talk also does the postprocessing (upload) step.] Similar to variant 2, but the release encoder (minion) only mounts the /video/fuse/$room/ from each recording cube. The encoded and tmp files life on one minion, the secondary minions mount /video/encoded and /video/tmp from the primary minion. [Reason: It is not guaranteed that the minion which encoded a talk also does the postprocessing (upload) step.]
Line 115: Line 126:
  
  
-==== New example with case 1 and 5:+==== New example with systemd units and case 1 and 5:
  
 {{drawio>c3tracker:setup-variant-3.png}} {{drawio>c3tracker:setup-variant-3.png}}
Line 124: Line 135:
  
   sudo mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros   sudo mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
-  sudo systemctl start +  sudo systemctl start crs-recording-scheduler  # A 
-  sudo systemctl start +  sudo systemctl start crs-mount4cut            # B 
-  sudo systemctl start+  sudo systemctl start crs-cut-postprocessor    # C 
 +   
 +  # check if everything is running as expected – you might have to disable/stop the other CRS workers D-F 
 +  sudo systemctl status -n 0 crs-* 
  
 on master minion (in this example `storage.lan.c3voc.de`) on master minion (in this example `storage.lan.c3voc.de`)
 +
 ``` ```
 mkdir -p /video/fuse/jev22/{Ahlam,Bhavani} mkdir -p /video/fuse/jev22/{Ahlam,Bhavani}
 mount -t cifs -o password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam  mount -t cifs -o password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam 
 mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani
-cd /opt/crs/tools/tracker3.0/ + 
-sudo ./start screenrc-pipeline # with steps D, E, F+sudo systemctl start crs-encoding             # D-encoding 
 +sudo systemctl start crs-postencoding         # E-postencoding-auphonic 
 +sudo systemctl start crs-postprocessing       # F-postprocessing-upload 
 + 
 +# check if everything is running as expected – you might have to disable/stop the other CRS workers A-C 
 +sudo systemctl status -n 0 crs-* 
  
 ``` ```
 +
 //(ensure that samba is installed on this master minion aka storage)// //(ensure that samba is installed on this master minion aka storage)//
  
Line 150: Line 171:
 mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
  
- +```
- +
-```   +
        
        
  
-==== Old example with case 5 and 6:+==== Old example with custom screenrc and case 5 and 6:
  
 on recording cube, without intros either copy or mount the intros from their source   on recording cube, without intros either copy or mount the intros from their source  
Line 187: Line 206:
 cd /opt/crs/tools/tracker3.0/ cd /opt/crs/tools/tracker3.0/
 sudo ./start screenrc-encoding-only # only step E sudo ./start screenrc-encoding-only # only step E
 +
 ``` ```
  
  • c3tracker/setup.1672255878.txt.gz
  • Last modified: 2022/12/28 20:31
  • by andi