I don’t want to have a Java Appserver running on a QA site. It takes too much memory and isn’t stable enough. Especially if it’s running on VM with limited memory and CPU. So I want something that can run as a CGI or FastCGI or apache module (mod_perl, mod_php, mod_python).

So the field is narrowed down quite a bit:

Tinderbox

Tinderbox is Mozilla’s build system. What I know about it is your central server outsources the build to a “tinderbox” which I assume can also run tests. It’s written in perl. I don’t know how up to date it is, but the idea of a collection of VMs running tests for a project is appealing. Especially if the build/test machines aren’t the same as the project server. If tinderbox has this built in, it goes to the top of the list. Anything else, I’d have to cobble something together.

Buildbot

Buildbot is written in python. I think mod_python is stable enough, but if not, CherryPy could probably run it. This opens up remoting potential as well, but I’m going to guess it doesn’t have the distributed nature I’m hoping for from Tinderbox. Since I haven’t heard much about Buildbot, I’m skeptical it will have all the features I might want.

Cerberus

I saw one review that said Cerberus doesn’t have a web interface. That might not be a deal breaker, but it makes it less likely to get a quick trial. Maybe Cerberus + CI::Reporter and ant JUnitHTMLReports would work. I’m also uncertain if Cerberus has a daemon. Somewhere it said it runs via cron, but that might be out of date. I think Luntbuild might do something similar. That might be acceptable, have cron run every 10 minutes and check for checkins and non-blocking. Or instead of cron-triggered have it checkin triggered via a SVN hook. But that makes reporting a bit more difficult, since something would apparently have to poll for the cerberus reports as well.

I’d love to hear any feedback.  Corrections, implementations, or other tools.