c3tracker:setup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Next revisionBoth sides next revision
c3tracker:setup [2023/09/02 09:07] – kunsic3tracker:setup [2024/10/19 16:50] – better document the "decentralised samba" setup stuebinm
Line 9: Line 9:
 ==== optimal properties from current project ==== ==== optimal properties from current project ====
  
-``` 
-Meta.Acronym                    camp2023 
-Meta.Album                      Chaos Communication Camp 2023 
-Meta.License                    Licensed to the public under http://creativecommons.org/licenses/by/4.0 
-Meta.Year                       2023 
  
-Processing.Auphonic.Enable      no +  Meta.Acronym                    camp2023 
-Processing.BasePath             /video/ +  Meta.Album                      Chaos Communication Camp 2023 
-Processing.MasterMe.Enable      yes +  Meta.License                    Licensed to the public under http://creativecommons.org/licenses/by/4.0 
-Processing.Path.Intros          /video/intros/camp2023 +  Meta.Year                       2023 
-Processing.Path.Outro           /video/intros/camp2023/outro.ts+   
 +  Processing.Auphonic.Enable      no 
 +  Processing.BasePath             /video/ 
 +  Processing.MasterMe.Enable      yes 
 +  Processing.Path.Intros          /video/intros/camp2023 
 +  Processing.Path.Outro           /video/intros/camp2023/outro.ts 
 +   
 +  Publishing.Upload.SkipSlaves    speedy,tweety,blade1,blade2,blade3,blade4 
 +  Publishing.UploadTarget         releasing.c3voc.de:/video/encoded/camp2023/ 
 +  Publishing.Tags                 <additional tags> 
 +  Publishing.Voctoweb.Enable      yes 
 +  Publishing.Voctoweb.Path        /cdn.media.ccc.de/events/camp2023 
 +  Publishing.Voctoweb.Slug        camp2023 
 +  Publishing.Voctoweb.Thumbpath   /static.media.ccc.de/conferences/camp2023 
 +  Publishing.YouTube.Category     27 
 +  Publishing.YouTube.Enable       yes 
 +  Publishing.YouTube.Playlists    <meep> 
 +  Publishing.YouTube.Privacy      <one of: public, unlisted, private> 
 +  Publishing.YouTube.Token        <meep> 
 +   
 +  Record.Container                TS 
 +  Record.EndPadding               300 
 +  Record.Slides                   yes 
 +  Record.StartPadding             300 
  
-Publishing.Path                 /video/encoded/camp2023/ 
-Publishing.Upload.SkipSlaves    speedy,tweety,blade1,blade2,blade3,blade4 
-Publishing.UploadTarget         upload@releasing.c3voc.de:/video/encoded/camp2023/ 
-Publishing.Voctoweb.Enable      yes 
-Publishing.Voctoweb.Path        /cdn.media.ccc.de/events/camp2023 
-Publishing.Voctoweb.Slug        camp2023 
-Publishing.Voctoweb.Tags <additional tags> 
-Publishing.Voctoweb.Thumbpath   /static.media.ccc.de/conferences/camp2023 
-Publishing.YouTube.Category     27 
-Publishing.YouTube.Enable       yes 
-Publishing.YouTube.Playlists    <meep> 
-Publishing.YouTube.Privacy      <one of: public, unlisted, private> 
-Publishing.YouTube.Tags         <additional tags> 
-Publishing.YouTube.Token        <meep> 
- 
-Record.Container                TS 
-Record.StartPadding             300 
-Record.EndPadding               300 
-Record.Slides                   yes 
-Record.StartPadding             300  
-``` 
  
 === Worker Filter Examples === Worker Filter Examples
  
-``` 
-EncodingProfile.IsMaster=no 
-EncodingProfile.IsMaster=yes 
-EncodingProfile.IsMaster= 
-Fahrplan.Room=Servus.at Lab 
-``` 
  
-Please note that the conditions in the "project to worker group" filter are currently evaluated with logical OR. +  EncodingProfile.IsMaster=no 
 +  EncodingProfile.IsMaster=yes 
 +  EncodingProfile.IsMaster= 
 +  Fahrplan.Room=Servus.at Lab 
 + 
 + 
 +Please note that the conditions in the "project to worker group" filter are currently always evaluated with logical OR. 
  
 Specifying a property with an empty value, which is often done for EncodingProfile.IsMaster, will match if this property does not exist at all on a ticket. So for EncodingProfile.IsMaster, specifying an empty filter will match on recording tickets which never have this property. Specifying a property with an empty value, which is often done for EncodingProfile.IsMaster, will match if this property does not exist at all on a ticket. So for EncodingProfile.IsMaster, specifying an empty filter will match on recording tickets which never have this property.
Line 60: Line 57:
  
  
-=== Simple: decentralised classic (Variant 2)+=== Simple: single-room setup (Variant 2)
  
 {{drawio>c3tracker:setup-simple.png}} {{drawio>c3tracker:setup-simple.png}}
Line 106: Line 103:
 The rest is very similar to above, but with different mounts so `/video/capture` is not hidden: The rest is very similar to above, but with different mounts so `/video/capture` is not hidden:
  
-```bash + 
-sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/fuse /video/fuse +  sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/fuse /video/fuse 
-sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/video/intros /video/intros +  sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/video/intros /video/intros 
-sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/tmp /video/tmp +  sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/tmp /video/tmp 
-sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/encoded /video/encoded +  sudo mount -t cifs -o uid=voc,password= //storage.lan.c3voc.de/encoded /video/encoded 
-```+
      
  
-=== decentralised pipeline (Variant 3)+=== decentralised pipeline aka "even more samba" (Variant 3)
  
 <panel type="danger" title="Attention">The "decentralized pipeline (Variant 3)" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust/improve the documentation here.</panel> <panel type="danger" title="Attention">The "decentralized pipeline (Variant 3)" should not be used by inexperienced users. Use the information above to find out how to get this variant working, then adjust/improve the documentation here.</panel>
  
-Similar to variant 2, but the release encoder (minion) only mounts the /video/fuse/$room/ from each recording cube. The encoded and tmp files life on one minion, the secondary minions mount /video/encoded and /video/tmp from the primary minion. [Reason: It is not guaranteed that the minion which encoded a talk also does the postprocessing (upload) step.]+Similar to variant 2, but extended to work with multiple rooms. Instead of using rsync, recorded snippets remain on the encoding cubes and ''/video/fuse/$event/$room'' are exposed via samba to the minions, while the encoded and tmp files live on one "central" minion; all other minions mount ''/video/encoded'' and ''/video/tmp'' from the primary minion [reasoning: the tracker cannot guarantee that the machine which encoded a talk also does the postprocessing (upload) step, so all minions have to see the same files]. 
 + 
 +Tracker filters have to be set only for the recording cubes, minions do not require any filters (but on smaller events without many minions, a ''EncodingProfile.IsMaster=yes'' filter can be a good idea, so sub formats won't crowd out the queues — they can always be encoded off-site later). 
 + 
 +On recording cubes: start the followin systemd units: 
 + * ''crs-recording-scheduler'' 
 + * ''crs-mount4cut'' 
 + * ''crs-cut-postprocessor'' 
 + 
 +On all minions, including the one acting as storage, do
  
-You have to set the room filters only for the recording cubes, the minions can process talks independently.+  mkdir -p /video/fuse/$event/{$room1, $room2, ..} 
 +  mount.cifs -o uid=voc,password= {//$encoder1.lan.c3voc.de,}/video/fuse/$event/$room1 
 +  mount.cifs -o uid=voc,password= {//$encoder2.lan.c3voc.de,}/video/fuse/$event/$room2 
 +  ...
  
-  * On recording cubes:   start systemd units for steps A ''crs-recording-scheduler'', B ''crs-mount4cut'', and C ''crs-cut-postprocessor''         +On all minions except the one acting as storage, also mount:
-  * On release encoders:  start systemd units for steps D ''crs-encoding'', E ''crs-postencoding'', and F ''crs-postprocessing''+
  
 +  mount.cifs -o uid=voc,password= //$storage.lan.c3voc.de/encoded /video/encoded
 +  mount.cifs -o uid=voc,password= //$storage.lan.c3voc.de/tmp /video/tmp
 +  mount.cifs -o uid=voc,password= {//$storage.lan.c3voc.de,}/video/intros
  
 +Finally on all minions, including the one acting as storage, start the following systemd units:
 + * ''crs-encoding''
 + * ''crs-postencoding''
 + * ''crs-postprocessing''
  
-==== New example with systemd units and case 1 and 5:+==== Old example with systemd units and case 1 and 5, which was used during jev22 in Munich:
  
 {{drawio>c3tracker:setup-variant-3.png}} {{drawio>c3tracker:setup-variant-3.png}}
Line 145: Line 160:
 on master minion (in this example `storage.lan.c3voc.de`) on master minion (in this example `storage.lan.c3voc.de`)
  
-``` 
-mkdir -p /video/fuse/jev22/{Ahlam,Bhavani} 
-mount -t cifs -o password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam  
-mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani 
  
-sudo systemctl start crs-encoding             # D-encoding +  mkdir -p /video/fuse/jev22/{Ahlam,Bhavani} 
-sudo systemctl start crs-postencoding         # E-postencoding-auphonic +  mount -t cifs -o password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam  
-sudo systemctl start crs-postprocessing       # F-postprocessing-upload +  mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani 
- +   
-# check if everything is running as expected – you might have to disable/stop the other CRS workers A-C +  sudo systemctl start crs-encoding             # D-encoding 
-sudo systemctl status -n 0 crs-* +  sudo systemctl start crs-postencoding         # E-postencoding-auphonic 
 +  sudo systemctl start crs-postprocessing       # F-postprocessing-upload 
 +   
 +  # check if everything is running as expected – you might have to disable/stop the other CRS workers A-C 
 +  sudo systemctl status -n 0 crs-* 
  
-``` 
  
 //(ensure that samba is installed on this master minion aka storage)// //(ensure that samba is installed on this master minion aka storage)//
Line 163: Line 177:
  
 on other minions on other minions
-``` 
-mkdir -p /video/fuse/jev22/{Ahlam,Bhavani} 
-mount -t cifs -o uid=voc,password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam  
-mount -t cifs -o uid=voc,password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani 
-mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded 
-mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded 
-mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp 
-mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros 
  
-```+  mkdir -p /video/fuse/jev22/{Ahlam,Bhavani} 
 +  mount -t cifs -o uid=voc,password= {//encoder1.lan.c3voc.de,}/video/fuse/jev22/Ahlam  
 +  mount -t cifs -o uid=voc,password= {//encoder5.lan.c3voc.de,}/video/fuse/jev22/Bhavani 
 +  mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded 
 +  mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded 
 +  mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp 
 +  mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
        
        
Line 186: Line 198:
  
 on master minion (in this example minion5) on master minion (in this example minion5)
-``` 
-mount -t cifs -o password= //encoder5.lan.c3voc.de/video/fuse/podstock2019/Aussenbuehne /video/fuse/podstock2019/Aussenbuehne 
-mount -t cifs -o password= //encoder6.lan.c3voc.de/video/fuse/podstock2019/Innenbuehne /video/fuse/podstock2019/Innenbuehne 
-mount -t cifs -o password= //encoder6.lan.c3voc.de/video/intros /video/intros 
-cd /opt/crs/tools/tracker3.0/ 
-sudo ./start screenrc-pipeline # with steps D, E, F 
  
-```+  mount -t cifs -o password= //encoder5.lan.c3voc.de/video/fuse/podstock2019/Aussenbuehne /video/fuse/podstock2019/Aussenbuehne 
 +  mount -t cifs -o password= //encoder6.lan.c3voc.de/video/fuse/podstock2019/Innenbuehne /video/fuse/podstock2019/Innenbuehne 
 +  mount -t cifs -o password= //encoder6.lan.c3voc.de/video/intros /video/intros 
 +  cd /opt/crs/tools/tracker3.0/ 
 +  sudo ./start screenrc-pipeline # with steps D, E, F 
 //(ensure that samba is installed on this master minion)// //(ensure that samba is installed on this master minion)//
  
  
 on other minions on other minions
-``` 
-mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/podstock2019/Aussenbuehne  
-mount -t cifs -o password= {//encoder6.lan.c3voc.de,}/video/fuse/podstock2019/Innenbuehne 
-mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded 
-mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded 
-mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp 
-mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros 
-cd /opt/crs/tools/tracker3.0/ 
-sudo ./start screenrc-encoding-only # only step E 
- 
-``` 
  
 +  mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/podstock2019/Aussenbuehne 
 +  mount -t cifs -o password= {//encoder6.lan.c3voc.de,}/video/fuse/podstock2019/Innenbuehne
 +  mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded
 +  mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded
 +  mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp
 +  mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
 +  cd /opt/crs/tools/tracker3.0/
 +  sudo ./start screenrc-encoding-only # only step E
  • c3tracker/setup.txt
  • Last modified: 2024/11/04 13:34
  • by jtbx