HC's Capture the Flag site
Capture the Flag gameserver

NOTE: This feature is considered usable, but has not yet been tested in a large scale CTF. Bug and success reports are welcome.

Distributed testscript execution

If your CTF is big enough, you may run into performance problems on the gameserver, when too many testscripts have to be exectued simultaneously. Starting with version 0.72, the gameserver supports distributed execution of testscripts. This document describes how to set it up.

Set up the peers

A peer is a machine dedicated to executing testscripts. It runs a helper daemon listening for testscript execution requests.

Your testscripts should behave exactly the same way on every machine. Thus, it is recommended you use the exact same setup for every machine. If that is not possible, you should carefully test your setup in advance to hosting the CTF.

To run a machine as a peer, you need the dexec.py file, which is included in the gameserver package, and python2.5 or newer. python2.4 is not supported. dexec.py expects a directory scripts/ to reside in its CWD. dexec.py will allow all files residing in scripts/ to be executed remotely. The invocation syntax of dexec.py is:

  python2.5 dexec.py MAXPROCS ALLOWED_IP1 ALLOWED_IP2 ... ALLOWED_IPn

MAXPROCS specifies the maximum number of processes allowed to run simultaneously. ALLOWED_IPx specifies an IP that is allowed to connect to the peer.

Excluding some testscripts at a peer

If you do not wish a certain script to be executed on a peer, simply do not copy it to that peer. The peer will report failure to the gameserver when requested to execute that testscript, and the gameserver will look for other options to execute the testscript.

NOTE: dexec.py hashes the contents of scripts/. If you add files to that directory while rexec.py is running, you have to restart it if you want these files to be considered.

Configuring the gameserver

The gameserver maintains a list of peers found in control/peers. This file contains the IP/hostname of one peer per line. If you modify the file while the gameserver is running, you must issue 'r<ENTER>' at the gameserver console to reload the file.

The gameserver picks a random peer when it wants to execute a testscript. If the peer refuses to execute the testscript, the gameserver tries all peers in the list in order. If all peers refuse execution, the script is executed locally. If that, too, fails, the according service is marked as TIMEOUT.

Caveats

The maximum number of testscripts executed in parallel is limited by the number of worker threads of the gameserver (configurable through control/numworkers), including remotely running testscripts. Thus, you need to set that value accordingly high.

For example, if you want to run 10 testscripts (in parallel) locally, 20 on peer 1 and 40 on peer 2, you need 70 worker threads.

Note that all scripts that could not be executed remotely are automatically executed locally. Thus, if, in this scenario, both peers fail for whatever reason, the gameserver ends up running 70, not 10 testscripts in parallel. If the gameserver can't handle this, make sure your peers don't fail.

Modifying number of worker threads on the fly

This may be necessary when you run into load problems (reduce number of worker threads) or time problems (increase number of worker threads).

To spawn five additional worker threads, enter

+<ENTER>

at the game server console. To kill one worker thread, enter

-<ENTER>

To kill five worker threads, enter

-----<ENTER>

at the game server console.

Note that a worker thread dies only after the test script controlled by it finished running. If you have 40 worker threads running, and issue

--------------------++++<ENTER>

at the game server console, you end up running 60 worker threads for a short time, not 40, as you might except.

---