Writing a CTF service may sound easy, and in fact it is. Writing a good CTF service is not. Some of the most important points to pay heed to when writing a CTF service are outlined below.
A CTF service, as the name indicates, must provide some services over a network, usually a tcp/ip network. The service contains arbitrary flaws that result in security vulnerabilties, incorrect behavior, and hard to understand code.
Usually, to be able to properly score the fixing and exploiting efforts of the CTF participants, so called flags are distributed to and later collected from the services.
Flags represent sensitive data. If you gain access to sensitive data that does not belong to you, you score offensive points by sending that data (i.e., the flags) to the gameserver. Flags can be "stolen" by guessing their IDs. IDs can be guessed by either, well, guessing, or by finding and using a flaw in a service. Sometimes, the latter method provides access to the flags itself. Flag IDs can be unique to flags, or unique to teams. Services that accept flags but do not associate them with some sort of "secret" have no way to provide the flags to the legitimate user without being cracked.
A service's flag storage/retrieval facilities should not be separate from the service's main functionality, because, as outlined, flags represent sensitive data the service handles. If the service does something big, and besides that also handles flags, why does it handle flags in the first place? Do not design your service to handle flags. Design it to handle sensitive data, and let your testscript provide sensitive data in the form of flag ids + flags.
Services should be fairly complex. Nasty code is hard to hide in simple services. Writing some ugly code is ok; let the participants clean it up.
Contrary to normal software development, you can begin writing a CTF service by deciding which language(s) to write it in, then deciding some application you want to implement. Of course, you can also do it the other way round. Usually, there's a CTF theme, which is intended to give you an initial idea for a service. Once you've got an idea, design your services and implement it.
The chosen language(s) do not have to be well-suited for the task you intend to do. On the contrary, hacking an smtp-server in TeX, or writing a cgi script in sed is cool and makes the CTF more interesting. However, do not assume your service is hard to fix because few people know the language it is written in. Make your service hard to exploit and hard to fix by making it complicated.
The first implementation should be bug-free and without design flaws. Test the first implementation thoroughly. Write a testscript, then test the service with the testscript. Deploy the service and the testscript to different systems and ensure they work without problems there.
When you're sure your service runs stable, implement the bugs. Always keep a bug-free copy. Bugs -- especially vulnerabilities -- should be in your code, not in some program you call or in some library you use. Document all your vulnerabilities. Be aware of their impact; hard-to-find vulnerabilities should give the attacker greater control over the service than easy-to-find vulnerabilities.
Some things to pay attention to when coding a CTF service and testscript:
This cannot be stressed often enough. The more complex your code is, the better you can hide the vulnerabilities. Also, writing complex code will make your service more interesting in general, and harder to replace by another implementation.
It is cool and acceptable to implement a standard service like an irc server, a webserver or a mail server. However, if you want to prevent the teams from replacing your implementation with a generally available one (which you do), you should violate a part of the standard(s) describing the service you're implementing at some point(s).
Debug output should not be visible to the teams, of course. But there's nothing worse than writing a service that cannot be debugged properly. Both your testscripts and services should provide debug output to the service author and the CTF organizer.
Your service will run in a different environment than the one you used for developing. The testscript will run in a different environment. Without proper debug code, it will be hard to find problems caused by subtle differences of the systems.
Your service must work stable. While this may sound counter-intuitive, it is crucial for a fun CTF. The teams do not want to re-implement your service because it is so poorly designed it crashes all the time. The teams want to audit the code to find well- (and not-so-well-) hidden vulnerabilities.
On the other hand, doing something like adding a mutable element to a binary tree is a great idea, and usually pretty hard to find :-) But that's a deliberate bug, not the result of poor design and/or testing. And yes, you should add something like this after you successfully stress-tested your service. Yes, that's extra work.
Most people lose interest in a service if they cannot find bugs no matter how hard they try. Add some easy ones to re-increase their motivation ;-)
The more complex your testscript is, the harder it will be for teams to fool your it. Teams try to remove functionality from services, or even replace it with a complete dummy service only capable of storing and retrieving flags. The more complex your testscript is, the harder it will be to fool it.
Wrong. If your coding abilities are not strong enough for you to believe you can write a functional service in the first place, do not write one.
It is crucial that your service is stable, except where you deliberately implement weaknesses. It does not work the other way round. Most instabilities you introduce by bad design cannot be fixed during the CTF, but they will make your service fail during the CTF, mostly at times and under circumstances you did not consider.
It is ok to implement bugs in a service that are not security relevant, but these must be well thought of. Do not leave something unfinished because of laziness or lack of knowledge. It confuses the teams, and sometimes cannot be fixed without modifications to the testscript. It causes a lot of frustration.
If your service does not work properly sometimes, but seems to be working correctly most of the time, throw it away. Especially if you cannot find the reason why it sometimes fails. Having a service that sometimes fails will cause unfair rating of the teams -- you will have a winner and a loser even if no one touches each other's services.
Having unstable services in a CTF is one of the most annoying things that can happen. Teams are going to be confused when one of their services is all of the sudden declared broken, when they did not touch anything. Spare yourself and your teams the hazzle. Write clean code.
This is acceptable as long as the bugs you introduced are well known to you. Never expect teams to fix things you left unfinished because of lack of knowledge or lazyness.
No, it does not. The scoring bot is yet another user of your service. You should not assume the bot knows special passwords or otherwise has privileges normal users do not have.
The purpose of the scoring bot is to check whether the service is usable for a normal user. All functionality exposed to the scoring bot must be exposed to any user. The only thing the bot knows the other users of the service do not know are the flags the bot delivered to the service. Every user may store and retrieve flags (which the bot will know nothing about). Obviously, only the bot's flags matter for defensive grading.
Check out my CTF game server for a more information on how to implement testscripts
$Id: writing_services.html 548 2009-07-23 20:58:53Z root $ Impressum