HOWTO: Unit testing Linux May 22nd, 2008
Mozilla’s evangelist and resident all-around-good-guy Chris Blizzard wrote an excellent piece yesterday on the dangers of hand-picking patches in a linux distribution, at least as they pertain to Mozilla and Firefox. I don’t need to say more on that as he covers it pretty well. I will just say that it’s a hard problem and difficult to get it right without having the entire big picture spread out in front of you.
I’m writing this in the hopes that we can make the process a little easier for those of you responsible for packaging Firefox (and hopefully other Mozilla apps down the road) on Linux.
First, the background: We do automated unit testing. We do it on every checkin and you can see the results posted to http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox. The machines responsible for producing those results are running buildbot. We’re currently using, a slightly customized version 0.7.5 (available from Mozilla’s cvs repo at cvs-mirror.mozilla.org in mozilla/tools/buildbot, branch BUILDBOT_BRANCH_0_7_5) on top of Twisted 2.4.0. Soon, we’ll be moving forward to version 0.7.7 and Twisted 8.x on all our machines as part of a Grand Unified Vision. More on that later.
We have some scattered documentation around the mozilla wikisphere that we’ll be consolidating and cleaning up over the summer. The important bits are: the main Mozilla Buildbot Wiki page at http://wiki.mozilla.org/Buildbot. Of some but decreasing importance is the BuildbotTestFarm setup page. It has links to the important buildbot master configuration and setup files, namely master.cfg and mozbuild.py in the unittest directory. Also, http://wiki.mozilla.org/BuildbotTestFarmUse has a really awesome diagram that I created (and recently updated!) that you’ll want to show to all your friends and possibly print out and hang on your cubicle wall.
Ok, let’s use all this to setup an example buildbot master and slave on your linux environment of choice. First, we’ll assume you’re capable of actually building the browser on your slave machine according to Linux_Build_Prerequisites. On your buildbot master (it can be the same machine as the slave if you’re just doing a single builder), create the master directory by typing “buildbot create-master unittest_master” (where unittest_master is your directory name, it can be anything). Copy master.cfg, mozbuild.py and the default mozconfig file into this newly-created directory and modify the .cfg file to taste. If you’re doing a straight copy and paste from our version, you can delete all those extra machine names and will have something that will look more like this: master.cfg. Rename URLs and links as appropriate. If you don’t use Tinderbox server for hosting your builds, you can remove the “c['status'].append(tinderbox.TinderboxMailNotifier(…” section entirely and just use the buildbot waterfall.
Once you’ve got that setup, start it up by running “buildbot start unittest_master” in the parent directory and you should see some log chatter as it starts up. You can verify by visiting localhost:2005 on the master machine and seeing a waterfall with an empty table in it. It’s waiting for a slave to connect. If you don’t see this, you’ve done something horribly wrong and will need to start from scratch, probably after thinking about it for a few minutes and maybe getting a cup of your favorite caffeinated warm beverage. This thoughtful reflection is important. I can’t stress this enough!
Now that you’re properly caffeinated, let’s create the slave. I’m going to assume you’re running it on the same machine as the one you’ve just created the master on. In a nearby directory, type “buildbot create-slave slave-dir localhost:9989 linux 1h34rtL1nuX”. It should create a directory named “slave-dir” and add a few files to it. Now the fun part: type “buildbot start slave-dir” and you should be up and running. Verify this on the buildbot waterfall page where you should see the slave connected and a build starting up.
At this point, I’m going to finish up with a few caveats. These instructions assume you’re using the same buildbot version we’ve used for these setups (version 0.7.5 running on twisted 2.4.0 with python 2.4 or 2.5). You’ll need to modify your master.cfg file slightly if you’re using a newer version. For excruciating details on how we setup these machines, see http://wiki.mozilla.org/ReferencePlatforms/Linux-CentOS-5.0. All machines running Reftests are required to have 24 or 32bit displays for image comparison. Lastly, the test machine will have to have run any version of Firefox before for the default profile to have been initialized.
The file “mozbuild.py” mentioned above contains the interesting bits. If you’re curious about how to run any of the individual unittests from a run, be sure to take a look in there at the “command=” parameters to the various testing classes. Good luck, and have fun!
Tags: Build, buildbot, howto, linux, Mozilla, Testing, unittest
Posted in Mozilla, Testing | Comments (1)
May 23rd, 2008 at 5:10 pm
[...] Campbell: Unit testing Linux. A great companion article to the one above, this time looking at Chris Blizzard’s warning [...]