C3 Ticket Tracker

The C3 Ticket Tracker is the central managing component in our postprocessing pipeline, releasing the final recordings. The tracker is a PHP web application and distributes tasks in form of 'tickets' to various workers. The workers – written in Perl, Python, or PHP – advance the state and update the ticket's properties via tracker's XML RPC API. The core system was initially written by FeM for 27C3 in 2010, and was released under the Apache License 2.0 end of 2017. Our instance is deployed at https://tracker.c3voc.de.

source code:

• https://github.com/crs-tools/tracker
• workers
  • https://github.com/crs-tools/crs-scripts (Perl)
  • https://github.com/voc/publishing (Python)
  • https://github.com/crs-tools/tracker-lib-php (PHP)
• custom fuse filesystem driver used by tracker workers
• (VOC) encoding profiles: TODO

Further resources:

• FeM tracker Beschreibung / FeM *C3 Doku
• FeM tracker doku
• old FeM tracker doku (probably outdated but still usefull

export VOC_WORKSPACE=~/voc
export VOC_CUTTING_TOOL=shotcut

function voc_check() {
	if [ -z "$0" ]; then
		echo "usage: ${FUNCNAME[0]} ID"
		return 1
	fi
	if [ ! -L "$VOC_WORKSPACE/check" ]; then
		echo "run voc_eventprep first!"
		return 1
	fi
	cd ~/voc
	mpv "check/${1}-hd.mp4"
	cd -
}

function voc_cut() {
    if [ -z "$0" ]; then
        echo "usage: ${FUNCNAME[0]} ID"
        return 1
    fi
    if [ ! -L "$VOC_WORKSPACE/cut" ]; then
        echo "run voc_eventprep first!"
        return 1
    fi
    cd ~/voc
    case "$VOC_CUTTING_TOOL" in
        shotcut)
            cp cut/**/${1}/project_shotcut.mlt /tmp
            ln -vs "$(realpath cut/**/${1}/uncut.ts)" /tmp
            shotcut /tmp/project_shotcut.mlt
            rm /tmp/uncut.ts
            cp /tmp/project_shotcut.mlt cut/**/${1}/
            rm /tmp/project_shotcut.mlt
            ;;
        kdenlive)
            kdenlive cut/**/${1}/project.kdenlive
            ;;
        *)
            cd -
            echo "invalid cutting tool configured"
            return 1
    esac
    cat cut/**/${1}/*frame
    cd -
}

function voc_eventprep() {
	if [ -z "$1" ]; then
		echo "usage: ${FUNCNAME[0]} acronym"
		return 1
	fi
	if [ ! -d "$VOC_WORKSPACE" ]; then
		mkdir -p "$VOC_WORKSPACE"
	fi
	rm -f ~/voc/{cut,check}
	ln -s "/video/fuse/$1" ~/voc/cut
	ln -s "/video/encoded/$1" ~/voc/check
}

function voc_mount() {
	if [ ! -d "/video" ]; then
		echo "/video missing, please create"
		return 1
	fi
	sudo mount -t cifs //storage.lan.c3voc.de/video /video -o password=dummy -o uid=$UID
	sudo mount -t cifs //storage.lan.c3voc.de/fuse /video/fuse -o password=dummy -o uid=$UID
}

see cutting

• Usage video YT link (german)

• TODO: CCCamp15 Recording / Postprocessing integrieren

see checking

Ticket types and states

• Parent tickets for each Fahrplan entry
• one recording ticket for each parent
• several encoding tickets, one for each format – e.g. MP4 HD, MP4 SD, WebM, MP3, Opus

• Create tracker project (you can also clone a matching project to have prefilled properties)
• Import schedule.xml
• Set all master tickets from staging to staged via mass-edit
• Adjust project properties, recommendations: (full reference at the bottom of the page)
• Compare project properties with previous instalment of the same event

optimal properties from current project

Project
Project.Slug			eh19
Processing
Processing.Loudnorm.Enable	yes
Processing.BasePath		/video
Processing.Path.Intros		/video/intros/eh19
Processing.Path.Outro		/video/intros/eh19/outro.ts
Publishing
Publishing.Path			/video/encoded/eh19
Publishing.Upload.SkipSlaves	speedy,tweety,blade1,blade2,blade3,blade4
Publishing.UploadOptions	-i /video/upload-key
Publishing.UploadTarget		upload@releasing.c3voc.de:/video/encoded/eh19
Publishing.Voctoweb.Enable	yes
Publishing.Voctoweb.Path	/cdn.media.ccc.de/events/eh2019/
Publishing.Voctoweb.Thumbpath	/static.media.ccc.de/conferences/eh2019
Publishing.Voctoweb.Slug	eh19
Publishing.Voctoweb.Tags	easterhegg, Wien, c3w
Publishing.Voctoweb.Url		https://media.ccc.de/v/
Publishing.YouTube.Category	27
Publishing.YouTube.Enable	yes
Publishing.YouTube.Privacy	public
Publishing.YouTube.Tags		easterhegg, Wien, bun intended, Chaos Computer Club Wien, c3w
Publishing.YouTube.TitlePrefix	Easterhegg 2019 -
Publishing.YouTube.Token	1/XXXXXXXXXXXX
Publishing.Mastodon.Enable	yes
Publishing.Twitter.Enable	yes
Record
Record.Container		TS
Record.Slides			yes

During event setup of the pipeline, you have to decide if you want leave the MPEG TS snippets only on the recording cubes or also rsync them to a central storage:

The first variant is typically used for events with more than one room. For bigger events we use the dedicated storage server in the event server rack, for smaller events a USB connected hard drive to one of the minions might be sufficient. Each recording cube exposes the files via rsyncd, which are pulled by an rsync process running inside a screen on the storage pc. For each encoderX open a screen tab on the central storage:

cd /video/capture/<EVENT>/
while true; do rsync -av --bwlimit=10000 --append --inplace -t encoderX.lan.c3voc.de::video/capture/<EVENT>/ .; sleep 60; done

start tracker workers on storage

cd /opt/crs/tools/tracker3.0
sudo ./start-screenrc-scripts.sh

Don't forget to the encoding tab and activate the encoding worker if you want the storage pc to also do some work.

To allow the encoding workers to do there job, they need to mount several folders on each minion:

sudo -s
mount -t cifs //storage.lan.c3voc.de/video /video
mount -t cifs //storage.lan.c3voc.de/tmp /video/tmp
mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded

The first mount point is read-only, the other two are read-write.

start tracker encoding workers

cd /opt/crs/tools/tracker3.0
sudo ./start-screenrc-encoding-only.sh

At small events, when the talks are finished for today, you can use the cubes to encoder master MP4 files. First: Stop voctocore.

The rest is very similar to above, but with different mounts so /video/capture is not hidden:

sudo -s
mount -t cifs -o password= //storage.lan.c3voc.de/video/fuse /video/fuse
mount -t cifs -o password= //storage.lan.c3voc.de/video/intros /video/intros
mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp
mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded

The second variant is only practical if you have at least one release encoder (aka Minions) for each recording cube. When using this variant with multiple rooms, you also have to set room filters in the tracker worker queues.

For every worker:

• set room filters in tracker

For every recoding cube:

• start tracker workers

cd /opt/crs/tools/tracker3.0

sudo ./start-screenrc-scripts.sh

For each minion:

• mount filesystems from encoder cube

#!/bin/sh
e=echo
if [ ! -d /video/tmp ]; then
  $e mount -t cifs -o password= //encoder$1.lan.c3voc.de/video /video
  $e mount -t cifs -o password= //encoder$1.lan.c3voc.de/tmp /video/tmp
  $e mount -t cifs -o password= //encoder$1.lan.c3voc.de/encoded /video/encoded
else
  echo "already mounted"
fi

• start tracker scripts for encoding

cd /opt/crs/tools/tracker3.0
sudo ./start-screenrc-encoding-only.sh

Similar to variant 2, but the release encoder (minion) only mounts the /video/fuse/$room/ from each recording cube. The encoded and tmp files life on one minion, the secondary minions mount /video/encoded and /video/tmp from the primary minion. [Reason: It is not guaranteed that the minion which encoded a talk also does the postprocessing (upload) step.]

You have to set the room filters only for the recording cubes, the minions can process talks independently.

Modify /opt/crs/tools/tracker3.0/screenrc-scripts:

• On recording cubes: comment out steps D-encoding, E-postencoding, and F-postprocessing
• On release encoders: comment out steps A-recording-scheduler, B-mount4cut, and C-cut-postprocessor
  

After modification, start the workers via the main screenrc on both recording cubes and release encoders:

sudo ./start screenrc-pipeline

on recording cube, without intros either copy or mount the intros from their source (here encoder6)

sudo mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
cd /opt/crs/tools/tracker3.0/
sudo ./start screenrc-pipeline # with steps A, B and C

on master minion (in this example minion5)

mount -t cifs -o password= //encoder5.lan.c3voc.de/video/fuse/podstock2019/Aussenbuehne /video/fuse/podstock2019/Aussenbuehne
mount -t cifs -o password= //encoder6.lan.c3voc.de/video/fuse/podstock2019/Innenbuehne /video/fuse/podstock2019/Innenbuehne
mount -t cifs -o password= //encoder6.lan.c3voc.de/video/intros /video/intros
cd /opt/crs/tools/tracker3.0/
sudo ./start screenrc-pipeline # with steps D, E, F

(ensure that samba is installed on this master minion)

on other minions

mount -t cifs -o password= {//encoder5.lan.c3voc.de,}/video/fuse/podstock2019/Aussenbuehne 
mount -t cifs -o password= {//encoder6.lan.c3voc.de,}/video/fuse/podstock2019/Innenbuehne
mount -t cifs //storage.lan.c3voc.de/encoded /video/encoded
mount -t cifs -o password= //storage.lan.c3voc.de/encoded /video/encoded
mount -t cifs -o password= //storage.lan.c3voc.de/tmp /video/tmp
mount -t cifs -o password= {//storage.lan.c3voc.de,}/video/intros
cd /opt/crs/tools/tracker3.0/
sudo ./start screenrc-encoding-only # only step E

User accounts can have different roles depending on what the user should be able to do:

• “User” - can cut and edit tickets
• “Restricted” - can only use already defined actions and can't define new actions
• “Superuser” - can edit profiles and (re)import fahrplan
• “Admin”

more details on https://github.com/crs-tools/tracker/blob/master/src/Config/AccessControl.php

Files that should be handled by the tracker (input files) should be named as follows:

  <room>-YYYY-MM-DD_HH-MM-SS.<extension>

Files need to be placed in directories according to the room name

Naming for intro and outro:

• Intro <frab-id>.dv
• Outro: outro.dv

The cutting process requires start and end marks. The tracker accepts frame numbers. If your player does not show frame numbers you can approximate with the formula

seconds * 25

assumed you have 25 frames per second. Note that this means you cut per second not per frame !!!