~robcee/ field notes of a pyroentomologist, now 33% snappier!

Mac Optimized Builds of Firefox

A couple of weeks ago I came across a reference to Mac Optimized Builds of Firefox. There’s a long tradition of community-based builds of Firefox for the Mac dating back to the PowerPC days.

A lot’s changed since those dark days, however. To say that we spend a lot of energy doing performance analysis is a bit of an understatement. We have entire teams dedicated to improving performance. Build options are analyzed and re-analyzed in an attempt to find cheap performance improvements. Specifying global code optimization settings can actually have an adverse effect on overall system performance because certain modules react better to, say, -O2 versus -O3, for example. All code that gets checked in has to make it through our performance tracking system or it gets backed out.

In the interest of Science, I wanted to actually see if there were any differences, so I pulled a fresh copy of mozilla-central and applied the referenced .mozconfig.

The first thing I noticed, was that –enable-profile-guided-optimization didn’t seem to be doing what I thought it would. Namely, I expected to see a browser window popup and run through some automation before starting a second compilation phase. It didn’t. Even so, the resultant build did feel a little faster at opening windows and tabs*.

So, I applied the latest stand-alone version of Talos to the build and executed the Ts (startup) and Twinopen (Window open) tests and compared this with the most-recent Mozilla nightly build of Minefield. Surprisingly, the windows didn’t appear to be opening much faster, but the startup time was on average 5% faster in the new “optimized” build vs. moz-central nightlies.

By this time, Ted had commented in the bug tracking all this and pointed me to the PGO page on MDC. It turns out, PGO builds require a separate make target “profiledbuild” instead of “build”. I reran the build, saw the automation was working this time and then reran the Talos tests. No change. Best guess around these parts is that GCC just isn’t very good at doing PGO. The same option gives us an across-the-board 10% speedup to all our tests on Windows.

It turned out that the option that was giving us the 5% startup time (and likely a slight boost to tab performance) was the 10.6 SDK target. Each revision of OS X includes optimizations and Snow Leopard is no different. If you want your own optimized build, just put that target (–with-macos-sdk=/Developer/SDKs/MacOSX10.6.sdk) in there if you’re on Snow Leopard and you’ll get a little extra snappy.

I haven’t booted into the 64 bit kernel to see if there’s any difference there, or even if the build options produce a 64 bit build. I don’t believe they do and the version shipped on the above optimized builds page don’t have the tell-tale “Open in 32 bit mode” checkbox in the app’s “Get Info” dialog.

* A word about perceived performance differences: It’s very hard to tell the difference between, say, 20ms and 30ms. That’s a 50% slow-down compared to 20ms but the actual slices of time are so small that the human brain has a hard time picking up on them. Nevertheless, people used to dealing with high-velocity timings (musicians, gamers, fighter pilots) can actually detect that kind of change. But there’s also a “wishful thinking” factor. It’s very easy to be told you’re going to see a speedup, and when given an identical application, think you’re seeing something faster. “It just feels snappier” you might say when given this placebo.

Firebug 1.5.0

I am very happy to be able to announce the release of Firebug 1.5.0 on addons.mozilla.org. This release represents a significant effort by the Firebug Working Group which saw the addition of some new faces over the last few months. Here’s a quick run-down of some of the new features:

• Enhanced Inspector
• More accurate Net panel timings
• Break on HTML mutation
• MathML and SVG namespace support
• Break on XHR
• Improved HTML editing
• Persist buttons on Console and Net panel
• Separate Computed CSS and Style subpanels
• Many many bugfixes and stability improvements

This is a huge release and these are just some of the highlights you’ll see in this new version. Please feel free to read the release notes and John Barton’s blog post for more details.

As always, if you encounter any problems, don’t be shy about filing a bug! We’ll be following up with quick point-releases if and when you find issues.

Special shout-outs and thanks to Mike Ratcliffe for the Inspector improvements, Steven Roussey for HTML editor improvements, SVG and MathML namespace patches, Honza for being awesome and John Barton for his tireless contributions.

If you’re new to Firebug, you might want to check out my introduction to Firebug screencast.

Intro to Firebug Screencast

A couple of weeks ago, I made a short-ish screencast about some of the basic features of Firebug. I plan on making more of these in the near future highlighting some of the features of Firebug 1.5 and some of the other, lesser known things you can do.

Right click and download in .ogg. or .mpeg4 to see the video in full-size.

View-Source IS Good. Full-stop.

I saw a tweet this morning from Joe Walker linking to this article asking Is View-Source Good? from Alex Russell of Dojo fame and I had to write about it. It’s something I’ve been thinking about a fair bit lately, so his timing was excellent.

Open systems are better and lead to more productive coders. This is a wild assertion I’ll make with some cheek, mostly because I don’t think any exemplars from closed-system programming can really refute it, or will. Programmers who get into pure languages like Lisp and Smalltalk ramp up quickly and become more productive because they have full view of the system’s source code.

So too, the web.

Web-programming is often a cut and paste experience. I hear the term “cut-and-paste inheritance” used derogatorily too-often. While it’s true that in some cases, from a code-design perspective, just pasting blocks of code from one document to another can lead to a poorly-designed system, it is great for getting little pieces of code to work. It is the scientific method applied to programming. You see something that works in one place, you try copying it over and see what fails on your page. Continue.

One area where view-source breaks down is in the analysis of “what happens” with that source code. The Error Console in Firefox is a poor-excuse for a debugging mechanism and linked to the view-source window only by a hopefully connected error and a line number you can click on. Errors can come from any tab and are not necessarily associated with your code. There is no sense of iterative feedback. You have to hunt for it and usually fall into a pattern of clearing your console and reloading the page to see when the errors occur. This leads many developers to write their own logging function. Or when they’ve graduated to more advanced development, install Firebug which presents dynamic views of your source code in exciting (and sometimes surprising) ways.

Another mysterious failing of view-source is in a dynamically generated page. Something that uses document.write() or a bunch of appendChild() calls often doesn’t show correctly in view-source. A trick I learned recently to see these types of changes is to select some text and right click on it and “view-selection source”. Not exactly intuitive, but this will show you what your code generated.

I don’t think anyone can claim that view-source is a bad thing, though I have seen people try to get around it. Usually under the guise of “I have some important information or data that I need to protect”. I have seen people file bugs to add a mechanism to Firebug to prevent inspection of the underlying HTML on certain pages. I find these types of requests pretty funny, generally, and a little sad because people who think this way fundamentally don’t understand what the web is about and how it works and why it got to be the wonderful thing it is today.

It’s a fairly common practice to hide content behind a flash barrier to prevent people from getting their grubby paws on it. This creates a large swath of the web that you really have no ability to play with or learn from. Also, I think it’s fairly safe to say that learning to program in Flash is a more expensive proposition than learning to program for the web. You have to pay for tools, books and example code. If you want to make something that looks like this widget over here that has some wandering penguins on it, you have to figure it all out for yourself.

Alex mentions generative tools and libraries like GWT that create HTML and CSS styling as by-products of their execution and that these create a not-very useful rendition for view-source. This is true. Even PHP-generated HTML can be pretty disheartening to look at if you’re trying to figure out what’s going on in a web-page — its poor formatting and machine-generated constructs are not pretty to look at, but you can still sometimes glean some of the workings behind them. Having hacked around in Wordpress for a few years, understanding the way the PHP code generates a blog’s output is an interesting experiment to puzzle through. I wouldn’t have been able to do that without view-source.

Anyway, I think I’ve burbled about this enough. I love view-source. There, I’ve said it. It is one of the most valuable tools we have for understanding and learning about the web.

Firebug and the JIT

One thing we hear a lot from Firebug users is that Firebug slows down Firefox. When Firebug is active, particularly when you’ve enabled the Console/Script panels, some pages perform much more slowly. Enabling these panels turns on all of our debugging hooks, so some slowdown isn’t surprising, but what may surprise you is that, in order to get accurate debugging information, these hooks also turn off Firefox’ high-performance Javascript JIT compiler, even when Firebug is inactive. And now we have a fix for that.

First, a little terminology. Feel free to skip this paragraph if you consider yourself a master of Javascript internals or compiler run-time optimization makes your skin crawl. Tracing is the mechanism Firefox’ Javascript engine (aka, “SpiderMonkey”) uses to improve code execution performance. It provides a major speedup for code running in Firefox 3.5 and up, often an order of magnitude for certain types of operations. It is the basis of the JIT or Just-in-Time compiler. Without tracing, the JS engine can’t optimize code as well, leading to significantly slower execution.

I need to be clear here: If you have Firebug installed you are probably not getting fast Javascript. Firebug doesn’t have to be active on your current page. If you have the grey icon on your status bar, you have probably disabled the JIT. This is true if you have ever enabled the Console and consequently the Script panels and left them on. This is likely true for most recent versions of Firebug. The quick fix is to disable the Script and Console panels via the mini menu on their respective tabs.

Boris Zbarsky has been doing a lot of poking around in the belly of the JS debugger lately. Working in conjunction with John Barton, they have concocted fixes for the soon-to-be-released Firebug 1.5 and Firefox 3.6. We’re working to get testable versions out as quickly as possible, but I can say that I’ve tested development builds using the pages mentioned in the bug and can verify that these fix the problem. As a side-benefit, I got to watch Boris dissect the problem through gdb in an Xterm (yes, an Xterm) and Emacs and it was pretty impressive and fun, to boot.

When installing Firebug one of the first things most people do is enable the Console. It’s such a useful debugging tool. I myself have been using Firebug for over a year without JITted code. I probably never noticed the slowdown simply because I’m so used to running my browser this way and have pretty fast machines. After disabling the Console and Script panels there are a few pages that just load much much quicker. It’s kind of shocking.

Since the patch has landed on Firefox 3.6, I’m expecting either a beta or release candidate of that to come out shortly. I’m also hoping for another Firebug beta (or release candidate) later this week. In the meantime, if you’re not using them, I recommend disabling the Console and Script panels and find out what you’ve been missing and turn them on only as-needed. This will likely be the work-around for Firefox 3.5 users unless we decide to push this back to that branch but I have my doubts that this is something we would do.

Firebug 1.5: New Features Revealed!

As of this minute, Firebug 1.5 is sitting comfortably in its third beta and available for download. This version is shaping up to be our best release yet and initial reports have been very positive regarding its stability, UI improvements and new features. So let’s take a look at some of the new features.

Improved Net Panel accuracy

One of the problems with Firebug’s Net panel in the past has been inaccurate timings. Because Firebug is entirely written in JavaScript some network and UI activity could block Firebug during long operations and cause the timings displayed there to be less than accurate. This has finally been corrected with the landing of a new service called the http-activity-distributor. For more details on the implementation and use of the newly-improved Net panel, see Honza’s blog post on the topic.

New Break Functionality

In Firebug 1.4, we introduced the concept of “break-on-next” to the Script panel. This was a “pause” button sitting between the inspect icon and the Console tab. In 1.5, we’ve extended this concept to the Console, HTML and Net panels to allow more exciting types of breaks.

In the Console, we’ve replaced the mini-menu Break-on-Errors option with the pause button. The reasons for this possibly contentious change was it made for a more consistent use of menus and the break button. Now, to enable Break on Errors, select the Console panel and hit the pause button. You’ll see that familiar glow to indicate that it’s waiting for an error. Now whenever an error occurs on the page, you’ll be dropped into the script panel at the line where the error occurred.

The HTML panel’s break button is a little different. This is the Break-on-Mutation feature. When this is enabled, whenever a bit of JavaScript modifies an HTML element, you’ll be taken to the Script panel and the modifying code while be highlighted. Related to this, you should be able to see modified HTML occurring in real-time in the HTML panel with affected elements and attributes being highlighted as they change in the page.

Finally in the Net panel, the break button acts as a Break-on-XHR button. This is intended to help debug AJAX apps by allowing you to halt the debugger during an XmlHttpRequest send. As in the other break types, you’ll be transported to the script panel when an XHR object fires off its request and you’ll be given the option to copy the message.

John Barton and Honza have written a great interactive demo page describing these new features on getfirebug.com.

Mixed Development

• We made a few tweaks to the UI in this version. We replaced the “Off” label with a single “power” button (or window close button on Mac) as promised during the last release.

• Kevin Decker added the search panel originally intended for version 1.4 with some nice options

• Persist option on Console and Net Panel. Save your data!

• Improvements to the Inspector.

Still more to come. Between now and final release we plan on hunting down a few more bugs to make this even more stable. Feel free to download and give it a try.

extensions.checkCompatibility.version

I was surprised this morning when I did my update to today’s newest nightly of Namoroka and all of my extensions were disabled. I was extra-surprised when I tried enabling my extensions with Nightly Tester Tools and it didn’t work. Turns out that Mossop’s been busy implementing a patch for bug 521905.

Now for version 3.6b3 and above, you’ll have to set a per-application version in about:config that looks like this:

See Mossop’s post on the topic for the deep story.

Eventbug (alpha) Released

This is so good, I thought I’d repost this instead of just firing it out to Twitter.

Jan Odvarko and John Barton have been working hard on getting a Firebug extension together to track event listeners on DOM objects. The Firefox parts needed to make this work were tackled by Olli Pettay in heroic fashion. You can download the alpha release here: http://getfirebug.com/releases/eventbug/1.5/. You will need Firebug 1.5b1 and Firefox 3.7 alpha (nightly) for this to work.

But before installing, I recommend you read Jan’s blog-post:

Software is hard | Eventbug (alpha) Released.

Firebug Features Poll part 2 – The Unloved

This is the second part of the Firebug Features Poll (part 1 is here). This time, focusing on the answers to the question: What is your least favorite Firebug feature. The one you never use. I don’t know if the question was worded poorly or if people just felt like venting, but many of the responders didn’t limit themselves to just one thing. As a result, I had to modify my non-scientific methodology a little bit and count mentioned features as votes instead of just lumping the responses into a single feature bucket.

As in the previous poll, I added a Junk column for responses that didn’t really make sense. “I <3 Firebug!” is great to hear (and Firebug <3s you too!) but it wasn’t really useful for the purposes of this poll. Augmenting this column, I created another category for responses lamenting Firebug stability or specific missing features. This was tied for first place with the most votes.

The Winners

Third place – Tie!

• Search feature
• Console. Surprised some people dislike the Console, but two of you did. One person mentioned a lot of exceptions coming from the browser showing up in the Console lowering the quality of information. These can often be cut down by unchecking the “Show Chrome Errors” and messages options in the Console’s mini-menu.

Second place – Tie!

These have the dubious distinction of having the same number of responses as the Junk column.

• CSS panel
• DOM panel
• Profiler

First Place

• Script panel — Responses varied from “I don’t know how to use breakpoints” to users who prefer using the Console to do printf-style interactive debugging. Other respondents claimed the script panel was just too buggy.

caveat: I believe there is probably more than one type of user of Firebug. Some of them may not be using Twitter and this poll might not have gotten to them. The types of users who replied and said that their favorite feature was the HTML Inspector and live-editing of CSS are probably the same group of people who don’t make heavy use of the Script debugger. If you’re a heavy debugger, please let us know in the comments.

Honorable Mentions

• HTML node editor – If you double click on a tag in the HTML view, you’ll see the node editor. Clicking the “Edit” button or using the Escape key will get you out of it.
• Event logging – right click on a node in the HTML viewer and select “Log Events”. Now every mousemove, click and keystroke in that node will be registered in the Console. How can you not love that?

There were no votes(!) for the Net panel, so everyone clearly loves it and thinks it’s awesome.

Take Aways

• we need to work harder on bugfixes and stability. Spending a lot of effort on stability and usability would probably go a long way towards making Firebug a better, more pleasurable to use piece of software.

• The CSS and DOM tabs aren’t loved. Adding better navigation to the DOM page, or reworking it entirely might be useful. Getting rid of it and the CSS tab entirely could be another option but not one we’d consider doing without some very strong feedback from the community.

• The Profiler ties in with the Script panel pretty closely. If you’re not doing heavy JS debugging, you might not need the profiler. It’s a pretty special-purpose tool to measure a page’s JavaScript performance.

• Search feature was a bit of a surprise, but we’re improving that for version 1.5 so hopefully it becomes easier to use.

As before, please leave us your comments if you think we’re missing something.

Automated Screen Grab and Upload Version 2

Last February I posted a write-up on how to automate uploads of screen captures to a web directory for sharing with others. The idea was inspired by the popular Grabup software (which Google has now added a malware warning to so I’m not linking to it). Since then OS X 10.6(.1) has been released upon the world and Grabup users have discovered that because of the way Snow Leopard names its screenshot files, Grabup doesn’t work anymore. Maybe they’re fixing it, I don’t know. There are other free alternatives, namely TinyGrab, some fiddling with DropBox, and others. It seems to be everyone’s pet thing.

With that, I’ve updated my AutoGrabUploader script and made it Even Better™! The one thing it didn’t do before was automatically dump the URL of the screenshot into the clipboard for easy pasting. You had to select it manually from the webpage that appeared in the browser. I’ve fixed that so it now gets added to the clipboard as you’d expect.

The list of requirements are pretty much the same as last time:

• Mac OS X 10.6 – template’s been saved with newer version of Automator
• Some kind of web-enabled remote storage (I’m using “people.mozilla.org”, but if you’ve got MobileMe you could do this with an iDisk or some other WebDAV-enabled system)
• MacFUSE + SSHFS – only if you need to mount a folder through SSH!
• FastScripts Lite
• Grabup-selection-template 2 workflow.

Install MacFUSE and sshfs as before. See the notes on the MacFUSE project page for getting that working. I had to remove a library from the sshfs application bundle to make it go on my laptop.

Unzip and edit the Grabup Selelection template in Automator as below (click the image to see the full size version):

automator screen grab uploader on flickr

When you’re done, save it to your ~/Library/Scripts folder as something like “Grabup Selection”. I have two of these, one for Selection and the other to grab a whole window. Use the Type dropdown in the Take Screenshot action to select Window and resave (careful not to copy over the interactive version).

The last step is to assign it a hotkey in FastScripts. Open FastScripts’ Preferences panel and assign Grabup Selection to a hotkey. I use Shift-Cmd-6 for selection and Shift-Cmd-5 for full window. Right next to the default Shift-Cmd-3 and 4 for the Mac’s default screenshot shortcuts.

If you’ve made it through this, congrats! Let me know if you have any problems.