While that’s cooking, Firebug 1.4 is going into its 7th alpha with even more tasty goodness included in the Net panel — we’re considering renaming it to the Honza panel, or maybe just putting an icon of Honza’s face in there. It’s got that much Honza in it. Also, John Barton has added some new Script panel improvements and performance enhancements. Scrolling should be even smoother and there’s a new “break-on-next” debugging feature which, at the moment, as almost sure to break things in new and exciting ways. Feel free to try it out at the usual place: http://getfirebug.com/releases/firebug/1.4.

John Resig and Honza have been working on getting Fireunit ready for prime-time as well. Over the next month, I plan on integrating that with some revamped build scripts to get our unittest environment running.

Now I’d like to reintroduce the Firebug Development Blog. It’s been freshly reminted and the links should finally work again on the getfirebug.com website. Expect updates from the team about new features and usage tips as well as developer-related information like API changes. If you’re into Firebug, you’ll want to add that feed to your reader of choice.

• Firebug 1.3.0b2 is available on getfirebug.com
• Discussed Honza’s awesome improvements to the Net panel
• Discussed John Resig’s further efforts on FireUnit and NetUnit integration
• Starting to push new code to 1.4 branch, 1.3 for bugfixes and cleanup only.

That’s the short version.

The longer version includes some good discussion about the net panel improvements, including some ideas for color improvements and drawing MozAfterPaint events on top of time-lines. We’re starting to see some additional load events super-imposed on the network traffic now, so the term “Net panel” is probably going to be replaced with something like, Construction Timeline in a later version. I don’t know what that name will be yet.

Resig’s improvements to FireUnit include a test-runner mechanism for running through a set of chrome files containing unittests. He’s also making good progress on integrating Honza’s NetUnit tests which will include the httpd.js server for hosting locally stored tests. We’re getting that much closer to having a usable unittest solution.

That’s not to say that we have full unittest coverage yet. I spent a couple of days messing around last week trying to write some Script panel unittests but ended up scrapping them because I coded around the thing I was trying to test. It was educational at least. I’ve moved back to working on rewriting some of the packaging tools so we can have a saner, cross-platform mechanism for building the extension and stripping out all the debugging and tracing code. Hint: It will probably be written in a language named for a certain non-venomous snake.

Not mentioned in the bullets, John Barton has moved Chromebug into the google code repository we’re using for Firebug. Feel free to check it out and give it a whirl. We should be much closer to having Chromebug in a usable state where Firefox and extension developers can start using it.

Oh, and one last thing: For all you testers running nightlies, we’re going to bump the MaxVersion on Firebug to 3.1 so you can start re-enabling your compatibility checks. Note that Firebug 1.3 on Firefox 3.1 will be considered experimental and useful for testing purposes only. We know there are problems, but we need to track them down and fix them in Firebug 1.4. If you encounter a problem, do a search in our Issues database and if you don’t find it, file it!

Enter FireUnit! An extension for Firebug that aims to be able to test Firebug itself, and later on, perhaps be a useful framework for developing unittests for the web. It’s still very early on, but there’s a core emerging.

Installing FireUnit is currently a bit of a manual affair. Download the source code from the github repository and extract the downloaded zip or tar file to your local filesystem.

Open your profile/extensions directory for the Firefox profile that you use with Firebug. Create a new empty text file there with the name “fireunit@mozilla.com” (no quotes, of course). In this text file, add the full path to the FireUnit directory you unpacked in the previous step. E.g., “/Volumes/Data/Projects/FIREBUG/fireunit” in my case.

Test panel of FireUnit after running commandline.html

When you open/restart Firefox with this profile and open Firebug, you should now see a new “Test” panel in the list of tabs. Go to the location bar and type in / paste chrome://fireunit/content/test/commandline.html. You should see some flickering as FireUnit runs through the tests in that commandline.html. When done, you should see a bunch of passes listed in the Test panel (hopefully no failures!).

While I’m rambling about testing, I should mention that there will soon be a new alpha version of Firebug 1.3 (a4) available for download at getfirebug.com/releases/firebug1.3. This would be an ideal version for you to try out FireUnit on! For the more adventurous in the crowd, Chromebug 0.3a8 is R261 on https://fireclipse.svn.sourceforge.net/svnroot/fireclipse/trunk/FireclipseExtensions/chromebug.

Ok, that’s laying it on pretty thick, but I have a fond affection for a few of these machines. For instance, qm-xserve01 was the original Mac unittest Xserve and nothing else has been able to touch it for compile and test time. We recently dropped qm-xserve06 onto the farm and at best, it’s about 10 minutes slower end-to-end than xserve01. I don’t know why this is and it shouldn’t be – xserve06 is newer and sporting more recent processors and I believe a faster bus. See also qm-win2k3-01, our first physical box running Windows unittests. It’s been there since April 26th of last year, as Shaver once quipped, “qm-win2k3-01 taught me to love”. We’re still waiting for the t-shirts.

Why are there three of everything? We borrowed a trick from Talos. One of the difficulties of running a testing box is that you sometimes get funny results. There are a lot of moving pieces on these machines and we try to keep them as tight as possible, running as few extra components as we can on identical operating systems (across a single platform), but sometimes things just go a little funny. The thinking was having sets of three (triadic) machines would introduce some error-checking into the runs. One orange run out of three can be discarded. Two out of three and you might have something a little harder to reproduce. Three out of three and you close the tree and backout. I think that system works pretty well in most cases.

The one sad point we’ve really struggled with during this period is the linux platform. We’ve been running linux unittests on our Centos5 ReferencePlatform which is solid and well-maintained by Ben Hearsum and Nick Thomas. Running it on VMs introduces some variability that has been really hard to shake down. Tests involving timers are almost guaranteed to fail at some point. Sometimes, if disk access gets a little busy, we see errors in reftests loading some resources, usually pngs of even the smallest size. Then there are the random mochi* failures. These problems are hard to reproduce and even harder to diagnose, usually resulting in a shrug and someone shakily pushing the “force build” button on the waterfall page in the hopes that the problem will go away on the next build. It often does.

If you can shed any light on these on-going issues, we could use some help. Please see bugs:

• https://bugzilla.mozilla.org/show_bug.cgi?id=431745
• https://bugzilla.mozilla.org/show_bug.cgi?id=435064
• https://bugzilla.mozilla.org/show_bug.cgi?id=428865

Moving forward to mozilla-central, we’ve got a few machines chugging along there and I hear qm-win2k3-03 picked up its first real regression yesterday so things seem to be progressing nicely. Over the next few weeks, we’re going to give the Firefox3/Gecko1.9 machines a spit-polish, upgrading all of them to buildbot 0.7.7 and matching their installations to what we’re using on the new machinery. As for what the new machinery will look like, I have a sneaky suspicion it’s going to look a little bit like Talos.

I’m writing this in the hopes that we can make the process a little easier for those of you responsible for packaging Firefox (and hopefully other Mozilla apps down the road) on Linux.

First, the background: We do automated unit testing. We do it on every checkin and you can see the results posted to http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox. The machines responsible for producing those results are running buildbot. We’re currently using, a slightly customized version 0.7.5 (available from Mozilla’s cvs repo at cvs-mirror.mozilla.org in mozilla/tools/buildbot, branch BUILDBOT_BRANCH_0_7_5) on top of Twisted 2.4.0. Soon, we’ll be moving forward to version 0.7.7 and Twisted 8.x on all our machines as part of a Grand Unified Vision. More on that later.

We have some scattered documentation around the mozilla wikisphere that we’ll be consolidating and cleaning up over the summer. The important bits are: the main Mozilla Buildbot Wiki page at http://wiki.mozilla.org/Buildbot. Of some but decreasing importance is the BuildbotTestFarm setup page. It has links to the important buildbot master configuration and setup files, namely master.cfg and mozbuild.py in the unittest directory. Also, http://wiki.mozilla.org/BuildbotTestFarmUse has a really awesome diagram that I created (and recently updated!) that you’ll want to show to all your friends and possibly print out and hang on your cubicle wall.

Ok, let’s use all this to setup an example buildbot master and slave on your linux environment of choice. First, we’ll assume you’re capable of actually building the browser on your slave machine according to Linux_Build_Prerequisites. On your buildbot master (it can be the same machine as the slave if you’re just doing a single builder), create the master directory by typing “buildbot create-master unittest_master” (where unittest_master is your directory name, it can be anything). Copy master.cfg, mozbuild.py and the default mozconfig file into this newly-created directory and modify the .cfg file to taste. If you’re doing a straight copy and paste from our version, you can delete all those extra machine names and will have something that will look more like this: master.cfg. Rename URLs and links as appropriate. If you don’t use Tinderbox server for hosting your builds, you can remove the “c['status'].append(tinderbox.TinderboxMailNotifier(…” section entirely and just use the buildbot waterfall.

Once you’ve got that setup, start it up by running “buildbot start unittest_master” in the parent directory and you should see some log chatter as it starts up. You can verify by visiting localhost:2005 on the master machine and seeing a waterfall with an empty table in it. It’s waiting for a slave to connect. If you don’t see this, you’ve done something horribly wrong and will need to start from scratch, probably after thinking about it for a few minutes and maybe getting a cup of your favorite caffeinated warm beverage. This thoughtful reflection is important. I can’t stress this enough!

Now that you’re properly caffeinated, let’s create the slave. I’m going to assume you’re running it on the same machine as the one you’ve just created the master on. In a nearby directory, type “buildbot create-slave slave-dir localhost:9989 linux 1h34rtL1nuX”. It should create a directory named “slave-dir” and add a few files to it. Now the fun part: type “buildbot start slave-dir” and you should be up and running. Verify this on the buildbot waterfall page where you should see the slave connected and a build starting up.

At this point, I’m going to finish up with a few caveats. These instructions assume you’re using the same buildbot version we’ve used for these setups (version 0.7.5 running on twisted 2.4.0 with python 2.4 or 2.5). You’ll need to modify your master.cfg file slightly if you’re using a newer version. For excruciating details on how we setup these machines, see http://wiki.mozilla.org/ReferencePlatforms/Linux-CentOS-5.0. All machines running Reftests are required to have 24 or 32bit displays for image comparison. Lastly, the test machine will have to have run any version of Firefox before for the default profile to have been initialized.

The file “mozbuild.py” mentioned above contains the interesting bits. If you’re curious about how to run any of the individual unittests from a run, be sure to take a look in there at the “command=” parameters to the various testing classes. Good luck, and have fun!