~robcee/ more than just sandwiches

The System Works

This afternoon we closed the tree while sorting out a mochitest failure. It was hard to tell if the crash was real initially, as there had been some tweaking of the tinderbox output from the testbots, and some changes to the harness itself. After some investigation it was deemed to be a real bug and the tree was closed.

Turn-around time from identification of the bug to tree closure to fix was about 90 minutes. Everyone in #developers remained calm and there was no name-calling or crying. While it’s likely that this bug would have been discovered eventually, the fact that it was caught when it was checked in and immediately corrected is not a small point. It may have saved everyone hours of detective-work and struggling with CVS. Instead, it was a minor event.

In other news, the reporting format of the testbots has changed to a more compact layout. They’re now reporting as number of passes and fails on a single line, separated by a slash. Mochitests are reporting as passes/fails/todos.

Additionally, another harness is coming online and reporting as “chrome”. This will be used for testing XUL and browser chrome. It should be available on all platforms by the end of the week.

It would be even more compact to say “mochitest: pass” or “mochitest: 2 failures”.

true, but the number of passing tests is actually useful. We can see at a glance when people are checking in more tests with their code. Plus, it’s interesting to see these numbers over time.

Or, possibly more importantly, when people aren’t checking in tests with their code.