Archive for the ‘Code’ Category
FireUnit - the early years October 7th, 2008
One of our goals for moving forward with Firebug is to build a system to run automated unit tests for it. Currently, Firebug is modified, packaged and then tested by a small (but enthusiastic!) group of developers. When changes are made to the codebase, the area under development might be tested by the developer, but there’s no way of knowing if what you’ve done has broken some other part of the application. As a friend of mine is fond of saying, software is hard.
Enter FireUnit! An extension for Firebug that aims to be able to test Firebug itself, and later on, perhaps be a useful framework for developing unittests for the web. It’s still very early on, but there’s a core emerging.
Installing FireUnit is currently a bit of a manual affair. Download the source code from the github repository and extract the downloaded zip or tar file to your local filesystem.
Open your profile/extensions directory for the Firefox profile that you use with Firebug. Create a new empty text file there with the name “fireunit@mozilla.com” (no quotes, of course). In this text file, add the full path to the FireUnit directory you unpacked in the previous step. E.g., “/Volumes/Data/Projects/FIREBUG/fireunit” in my case.
Test panel of FireUnit after running commandline.html
When you open/restart Firefox with this profile and open Firebug, you should now see a new “Test” panel in the list of tabs. Go to the location bar and type in / paste chrome://fireunit/content/test/commandline.html. You should see some flickering as FireUnit runs through the tests in that commandline.html. When done, you should see a bunch of passes listed in the Test panel (hopefully no failures!).
While I’m rambling about testing, I should mention that there will soon be a new alpha version of Firebug 1.3 (a4) available for download at getfirebug.com/releases/firebug1.3. This would be an ideal version for you to try out FireUnit on! For the more adventurous in the crowd, Chromebug 0.3a8 is R261 on https://fireclipse.svn.sourceforge.net/svnroot/fireclipse/trunk/FireclipseExtensions/chromebug.
Tags: Code, Firebug, fireunit, Mozilla, Testing, unittest
Posted in Code, Firebug, Mozilla, Testing | Comments (2)
Building Chromebug August 29th, 2008
This week was a pretty incredible week for web stuff. The Labs people kicked off their Ubiquity prototype, letting the world get a taste for some of what will be possible through natural language processing and the browser. I also discovered a new feed reading add-on for Firefox called Feedly which does some very cool things with Google Reader to present web feeds in a new way. While reeling from all this new-found power, I had an intense meeting with John Barton who walked me through the setup procedure for Chromebug.
First, there are a couple of caveats about Chromebug. It’s still in an experimental state. Some parts work well, like the inspection of a chrome window’s DOM tree and CSS. Other parts are a little flakey or down-right broken. As of right now, the default build instructions will attach you to a 1.3 version of Firebug which has some improvements to handling large script files but is also somewhat broken. It should improve shortly.
So, with that out of the way, I’ll run through a quick Chromebug setup and install. The instructions are on the sourceforge page for Fireclipse, a plugin for Eclipse that adds some debugging capabilities for Firefox and/or JavaScript.
Step 1. From a suitable directory on your computer, check out the chromebug sourcecode from svn. There are external svn references to include two other add-ons, chromelist and firebug. These will get checked out as well.
Step 2. Create a new profile for Firefox using the Profile Manager. I called mine “Chromebug”. Make sure you start Firefox with this new profile to finish the creation process.
Step 2a. Make note of the location of this new profile. You’ll need the full path for the following step. On Windows, this would typically be in “C:\Documents and Settings\%USERNAME%\Application Data\Mozilla\Firefox\Profiles\xxxxxxxx.Chromebug\”. Replace %USERNAME% with your username, and xxxxxxxxxx with some randomly generated string. Just look for *.Chromebug in that directory. You will need this for the “install.dir” property in step 4 below.
Step 3. In a console (or terminal window), cd to the chromebug directory you checked out earlier. E.g., cd ~/Projects/fireclipse/extensions/chromebug
Step 4. Create a local.properties file using your favorite text editor in this directory. Add the following properties to it (each property should be on a single line, regardless of how this blog breaks it up):
firebug.dir=../firebug/branches/firebug1.3
chromelist.dir=../chromelist
Step 5. Create links to the chromebug extensions. This can be most-easily accomplished using the build.xml file provided in the fireclipse/extensions/chromebug directory and the Java build utility ant. If you don’t have ant installed, you can do this manually as per the instructions in the included readme file. It’s standard extension development linkage.
And now you should be ready to fire it up using (on windows):
Replace firefox.exe with firefox-bin on unixey OSes. You should see two windows open. One, a smaller window with the title Chromebug, the other, a standard browser window. Before you get too excited and jump in, you should quit the application and restart it (again, using the above command) to make sure the chrome files get registered properly. Now you should be ready to play!
Please keep in mind that Chromebug is experimental. There’s a lot of code in there and not all of it’s working. We are interested in getting more people using it and reporting problems though so check it out, and if you encounter any problems, feel free to come to #firebug in irc.mozilla.org and ask questions. You may have better luck using Firebug 1.2 instead of 1.3 currently, but I haven’t tested that myself. It should be a matter of changing the firebug.dir property in the local.properties file from step 5 above.
I’d like to thank John Barton for patiently walking me through the above and giving me a walkthrough of the code itself. It’s incredibly cool stuff and I’d love to see it become part of the standard arsenal of extension development tools as well as part of the toolkit for developing firefox front end features. In the coming weeks, I’ll post follow-ups on the state of chromebug and what you can do with it.
Happy debugging!
Tags: Build, Chromebug, Code, Firebug, Mozilla
Posted in Chromebug, Code, Firebug, Mozilla | Comments (5)
Talos machines going down for upgrades August 30th, 2007
The performance machines reporting to the Firefox and Mozilla1.8 trees will be going offline tomorrow morning, Friday, August 31, 2007 for some improvements to both buildbot code and support for the Linux and Mac ports of Talos. We are expecting the outage to be from around 5am PDT to 9am PDT*.
* assuming Nothing Bad™ happens
Posted in Code, Infrastructure, Testing | Comments (0)
AsynchronousMochiFixTinderbox June 21st, 2007
This morning, while no-one was looking, I secretly landed John Resig’s patch for bug 379506 (after a quick run-through in my local Linux VM). Everything is A-OK.
In other news, Axel Hecht submitted a nice patch for the TinderboxPoller in our local copy of Buildbot. I snuck that in too.
Thanks guys!
Posted in Code, Testing | Comments (0)
What a week June 8th, 2007
or month, actually… but I’ll try to keep this post limited to the events that occurred most recently.
The background: John Resig did some great work building a new test suite for a recent cut of the MochiKit test suite to run under mochitest (bug #379506). Shaver popped into #developers the other day and said something along the lines of, “could you help John with this review? KTHXBYE” Naturally, I said, sure!
I downloaded the rather lengthy patch and applied it to a pair of trees, did a build and ran runtests.pl. The new tests ran through flawlessly and, seemed ready to go after a quick update to a makefile that was out-of-date. I landed the patch.
And set the tree on fire. (linux, specifically)
Now a word about case-sensitive (or case-”preserving”) file systems. Mac and Windows (HFS and NTFS respectively) are not really case-sensitive. They will let you put differently-cased characters into file and directory names, and they will show them to you, but they don’t really know the difference between Mochikit and MochiKit.
Not so with Linux’ various UNiXey filesystems.
After excising the error in the Makefile, the redness and swelling cleared and Linux builds started happening again. Unfortunately, and this is where I ask for help, the MochiKit test suite didn’t run on Linux. The only info we had to debug it with was a cryptic:
*** 3605 INFO Running /tests/dom/tests/mochitest/ajax/mochikit/test_Mochikit.html…
*** 3606 ERROR FAIL | Test timed out. |
The MochiKit suite is still in there, merely removed from the Makefile for the time-being until we can figure this out. Astute readers might look at the above filename, and think, “hey, there’s a lower-case ‘k’ in there.” No luck, that’s the filename. I tried increasing the timeout on the tests with the very busy Rob Sayre’s instructions with no luck. Attempts to run the suite on a local Fedora Core 6 VM provided no help, failing to run MochiKit the first run through, then passing it on a subsequent run.
Apologies to Robert O’Callahan (roc) and the sheriffs for leaving the Linux machines in that state of orangeness overnight.
PS, Lesson learned: Check Linux first.
Posted in Code, Testing | Comments (2)
(left) Justified and Ancient March 26th, 2007
(with apologies to the KLF…)
I saw this note on Schrep’s blog this afternoon and remembered, that last week was actually kind of a milestone for me. With dbaron’s help. I made my way onto a 4-digit bug – the venerable #3247 - counters in css-generated content.
David Baron wrote the original tests (which you can see at http://dbaron.org/css2.1/tests/imported-20050702/ if you’re so-inclined). I wrote a little python to convert the tests into something that could be run with reftest a few months ago. I was looking for something to use with it and they seemed like a worthy candidate.
Fast-forward to last week, and TimR asked me about those tests I’d converted. So, I dug ‘em out, popped them on my people account and forwarded a note to David. After some cleanup (there were a couple of errors in there that my conversion conveniently included for us), they are now living happily at in http://lxr.mozilla.org/seamonkey/source/layout/reftests/counters/. You can see their results running on every checkin on the Firefox Tinderbox page.
PS, kudos to Kray2, Nickolay, Callek and others for the “Creating Reftest-based unit tests” page on MDC.
Technorati Tags: Code, Firefox, Quality, Testing
Posted in Code, Firefox, Quality, Testing | Comments (2)
it’s the little things February 7th, 2007
That little page was my first view of the new testing buildbots living in the Mozilla colocation facility. It may not look like much yet, but it’s overflowing with potential. What does it do, you wonder? It runs test-enabled builds, then runs make check, reftest and mochitest. From the three columns, you might infer that it does this for the three major platforms, Windows XP, Linux and Mac OS X — or at least it will when the Xserve arrives at the colo.
You can see the results reporting to the MozillaTest tree on tinderbox.mozilla.org. Currently, the two operational buildbots are “linref test” and “winxp test” (don’t worry, I have an exciting naming scheme all picked-out and ready for action – stay tuned!). Check out the logs for examples of what the make check, reftest and mochitest reports look like. If winxp is orange or burning, it’s because we still have a couple of issues to work out. Some of the reftests are failing on that platform and dbaron is writing a modification to the manifest language so we can mark individual tests as failures on specific platforms. The other, more troubling bug is a debugger prompt I can’t seem to get rid of. Neither setting XPCOM_DEBUG_BREAK to “warn” nor adding a registry entry for windbgdlg (thanks timeless!) seemed to work. I’m hoping the problem will just go away with the addition of more RAM, but that seems unlikely.
On my todo list, I have some work to do to get the different tests reporting rolled-up stats to the tinderbox. We’ll be able to see why a tree is orange instead of looking through the full logs. There are a couple of little profile writing things I can make better. At some point, soon, I’m going to work with Bob Clary on getting the JS tests automated and reporting under this system. Then there’s going to be some work getting Alice Nodelman’s performance testing framework running under it and reporting to the tinderboxes. I’m also trying to setup a leak-testing build to replace the Tinderbox that just stopped working and weaseled its way to the front of my to-do list. Ultimately, I’m hoping we can have these on a more visible tinderbox, but they need to be stabilized first. Oh, and I need to check these configuration changes in and flesh out the very skimpy docs I wrote while putting this together.
Whew, this blog post would’ve been much shorter if I’d just written it last week when I had that one little picture.
Hopefully, having regularly running and reporting tests will encourage developers to write more of them. Individually, unit tests are tiny and kind of uninteresting. When you’ve got thousands of them running constantly though, then you’ve got something.
PS, special thanks to everybody who helped out along the way to get this stuff running. To the IT crew (mrz, justdave, aravind, justin, …) for putting up with the constant bug-filing back-and-forth for VMs I needed or VMs I’d busted, the tireless efforts of the build team (Rob Helmer and J. Paul Reed) taking time away from their real jobs to setup the linux reference VM, Rob Sayre and Dave Baron for their excellent test frameworks, Jeff Walden for the js http server, Ben Hearsum for his buildbot work and Dave L. for getting the ball rolling many months ago. All the developers who wrote tests… I’m being played off the stage… thanks Mom!
Technorati Tags: Build, Computing, Firefox, Software
Posted in Build, Code, Firefox, Quality, Testing | Comments (1)
OS X fork: Resource temporarily unavailable July 28th, 2006
I ran into this today while trying to get a working build of XULRunner on OS X. I’ve seen this error before while doing browser builds but didn’t bother to do any searches on the source of the problem. Instead I just killed applications and dashboard widgets and went on my way. Today I decided I needed to fix it and dug up this little gem. Upping my maxprocperuid to 256 seemed to do the trick. Also, that is a really great picture of chrismiles.
Posted in Code | Comments (0)
Extensions for Automation July 21st, 2006
This week I’ve been taking a different tack on the testing problem. I’ve been thinking about unit testing at the C++ level over the past couple of weeks and have a couple of frameworks to try out. I’ve had a couple of issues with both of the frameworks which I’ll get to in a later post.
In the meantime, I’ve lifted myself out of the C++ and taken a look at Extension Development. There are a number of places in the application that can be tested with some simple extensions. Or so the theory goes… This week, I’m writing an extension to test some bookmarking functions. I’ll be documenting the process in detail on my wiki page for anyone interested.
One final note, Ted Mielczarek’s Firefox/Thunderbird Extension Wizard is a great starting point for building an extension.
Posted in Code, Quality | Comments (4)
robcee on Quality, Code and The Internet July 5th, 2006
Hello World. Welcome to my corner of it. Let me tell you a bit about myself and what I do.
I’ve got 10 years of experience in Object Oriented programming. My weapon of choice for most of that has been the Smalltalk programming language. I’ve worked as a consultant for government and private industry. I’ve also been a C programmer since learning the language and hacking on my Amiga way back in my high school years. I’ve done C++ and Java programming too. But enough about me…
One of the cool things about the Smalltalk community and the “eXtreme Programming” movement that came out of it was the push for better testing in code. In addition to some of the other tenets like, pair programming and iterative development, unit testing is one way to gather solid metrics on code quality and ensuring regressions don’t occur when new code is committed. With a unit testing framework, a developer can run the full set of tests and know quickly if any of their changes have broken something else in the code base.
a note on “units” — What is a “unit”? It can be a module, a class, a method or even a single block of code. Any encapsulation can be considered a unit and is up to the developer to define. A suite of tests can be written to test correctness of the unit and assert that a piece of code evaluates to true.
How can this work in the Mozilla Project? Unfortunately, some of the things which make unit testing so easy in a language like Smalltalk or Java are going to be more difficult in a large, compiled code base like Mozilla’s. It is not a simple matter to write a unit test, recompile the whole application and check your results. The turn-around time is too large. Worse, there are multiple platforms to run the tests on. A fully-passing test-run might not pass on another platform or even system configuration. While it might be possible to test some units in isolation, the majority of the tests will likely require a full rebuild of the app.
In order to allow testing of multiple platforms, we will need a testing farm, likely running on VMs with each reporting to a separate unit-test tinderbox.
Currently, I’m setting myself up with a development environment and investigating MinUnit. It is absolutely the lightest-weight-possible testing framework, but it’s totally open and easy to modify. Once that is in place, I plan on looking at CxxTest which got some positive review in this excellent article.
Once some proof-of-concept code is in place, I plan on working with the Zach and the build team on setting up our test servers and a tinderbox. Big plans. I will stop talking now and get to work…
Posted in Code, Quality | Comments (0)