I Ain’t Dead

You may have noticed that lately I’ve been somewhat absent from, uh, everything. Dead to the world for most of December, and just brief ghostly appearances so far in 2013. I’ve posted a couple of times on Twitter about why, but I wanted to write something a bit more substantial about it all, now that I’m able to do so. Thus begins my tale of woe and boredom…

Once upon a time (1st December 2012), in a land far far away (New Zealand), there lived a great bearded wizard (that’s me), tormented by an evil troll.

Old garden shed

Ok, so that’s not a troll — it’s our old garden shed. Although I do consider it to be slightly evil now.

Notice the grape vine creeping up the trellis; I was attaching the trellis to the wall of the shed one bright Saturday afternoon. I bent down to pick something up, stood up and turned around, and in a fit of bloody revenge the shed viciously attacked me. Or I may have just hit my head on the corner of the shed — I don’t specifically remember that part. I vaguely remember feeling well enough afterwards to finish the job and clean up; though afterwards I did need to retire inside for a rest on the couch, then have an early night.

Sunday morning was quite the opposite of “feeling well enough”. “Awful” is a much more accurate description. I’m usually very resistant to see any doctor, but I got there as soon as I could on Monday. The medical field seems to involve a lot of inexact science (or as I like to call it, “guessing”), and my doctor’s best guess was a pinched nerve in my neck and a concussion. I knew very little about concussions, but my doctor warned me to stay away from computer screens and books.

Now, I’ll admit to a fair amount of ignorance here, as at this stage I expected to have an easy (if somewhat boring) time and be back at work after a week. Obviously, that didn’t exactly go to plan.

In fact, I was so wrong that on Tuesday Anne and I planned to take a trip out of town (Anne driving, of course) so I wouldn’t be completely bored out of my mind. We got as far as the emergency department of the hospital. Nothing serious, but I did learn that even sitting as a passenger in a moving car was enough to bring on some very worrying symptoms. Confusion, disorientation, difficulty focusing attention, blurry vision, double vision, atrocious headache, nausea, etc. All of which turned out to be normal, for a concussion — but nevertheless scary to experience.

I think most people don’t have much of an understanding as to how a concussion (or any brain injury) can affect a person. I certainly didn’t. This lack of understanding, and the types of symptoms and limitations, can be very isolating. As it turns out, being a programmer involves doing everything that you’re not allowed to do when you have a concussion:

  • In front of a screen
  • Reading
  • Concentrating
  • Thinking

Yea, really — I was pretty much banned from thinking. And for good reason — thinking made my head feel awful. Interestingly, I was also banned from doing much physical exercise. I found I was very easily physically exhausted, as well as mentally exhausted. Conversations with people overloaded my brain very easily, and I couldn’t cope at all with listening to groups of people. Both the mental and physical symptoms are caused by what’s called neurogenic fatigue — which apparently is remarkably similar to sleep deprivation. And so the trouble with a concussion is that you can’t push yourself to recover faster — it just ends up making things worse.

So I was stuck for most of December trying my best to do as little as possible. I plan on writing a separate blog post about this, but it’s actually quite difficult to do nothing — really do nothing. I spent a lot of time sleeping, or laying on the couch or outside trying to keep my mind blank. Some forms of meditation worked, but many involve too much concentration. Most days I had a short trip into a quiet café somewhere — I’ve never been in cafés so often before in my life. Needless to say, it was a remarkably boring time. But thankfully this was during a very pleasant New Zealand summer, which permitted plenty of time lounging in the sun and quiet walks along the waterfront of St Clair beach (yea, such a hard life).

Oh, and of course it helped having my attentive friend Harley keeping me company:

Harley, our ginger and white kitty

On 18th December I was given the all-clear to start slowly returning to normalcy. On 3rd January I was told I could start reading work email for an hour, spread out over a day, every second day. It took me until 1st February to build up to being able to work half a day, albeit still not coding. If this seems slow, it was — agonisingly slow.

It’s 14th February now (for me, at least – yay timezones) and I’m still doing roughly half days, but building up what I’m doing. So I’m finally getting through my review queue, some technical reading, and generally catching up on things I missed. Hopefully I’ll soon be able to get back into some serious coding, help finish that little project I started last year, and generally be back into my normal routine. In the meantime, I’m a delicate little flower, so please don’t bury me with too much work :)

Add-ons Manager API: Change in event order when restartless extensions are installed

Over in bug 727398 I’m planning on making a change to restartless extensions that could potentially affect addon authors who use the Add-ons Manager API. This only affects code that registers a AddonListener or InstallListener event listener to get notified of add-on and install events, while also interacting with the startup of restartless extensions. I couldn’t find any add-ons on AMO that would be affected by this, but I thought it would be worth blogging about anyway.

Currently, when a restartless extension is installed, it’s startup() function is called before the onInstalled and onInstallEnded events are fired. This meant that a restartless extension would run it’s initialization code before anything else thought it was installed.

However, that causes some issues. For example, if a restartless extension uninstalled itself on startup[1], then it could be uninstalled before it was installed. Unsurprisingly, the Add-ons Manager UI broke when this happened (since it uses the same APIs that add-ons use).

So bug 727398 will change the order of those operations. When a restartless extension installs, the following will happen in this order:

  1. The extension’s install() method will be called
  2. onInstalled event will fire (AddonListener)
  3. onInstallEnded event will fire (InstallListener)
  4. The extension’s startup() method will be called

 

Update: This change will ship in Firefox update 14, which is scheduled to be released on 2012-07-17.

 

[1] Why would an addon immediately uninstall itself, you ask? Imagine an add-on whose sole purpose was to flip a preference to disable a newly-shipped but buggy feature, that would get re-enabled on the next application update. This is one of the things that a hotfix add-on could do. Additionally, I’ve been playing with various ideas for restartless extensions as a user support tool – the extension installs, does some work or collects some diagnostic data, then immediately removes itself.

The Add-ons Manager and I are rather good chums

Back in… er… a long time ago, I wrote an add-on called Filter Extensions. I was just scratching an itch – I had more add-ons than the Add-ons Manager was designed to cope with. Turns out people like add-ons.

Because of that little add-on, one of my first projects after I started working at Mozilla was teaching the Add-ons Manager about outdated plugins. The theory was that since I had worked on that add-on, I should already have some background in working with the Add-ons Manager code. Hah… Yea, no, that didn’t help at all. That feature eventually shipped in Firefox 3.6, which now feels like an eon ago.

With a whole two Add-ons Manager related things under my belt, I must have earned myself a reputation for really loving anything to do with the Add-ons Manager. So late in 2009 I was asked if I’d like to help Dave Townsend with the re-write of the Add-ons Manager that he had been planning for some time (with Jennifer Boriss Morrow for UX, and Henrik Skupin for QA). It would be just a short project, helping out with the new UI – nothing major, I’d be able to get back to finishing my other project soon enough. Turned out that at that stage there was no UI yet… and it would take about a year for me to write it. The first very basic iteration was written during my Christmas holiday in Motueka, a few days later I threw away half the code due to a change in the (still young) UI design. We eventually shipped the rewritten and redesigned Add-ons Manager in Firefox 4. By this stage, the Wiki page containing the notes from our weekly meetings was so long that it often took several minutes to load. And sometime during that adventure, Dave made me a peer for the Add-ons Manager module. I say “sometime” because I don’t actually recall him ever telling me.

Fast forward to a few months ago, where I got the chance to break a third of all the unit tests for the Add-ons Manager. Okay, maybe that part wasn’t so fun… but solving the add-on compatibility problem was.

Apparently that (and everything else) went well, because then this happened:

Dave: Hey, so, wanna be module owner?
Me: Yea, sure.
Dave: Oh… I expected to have to convince you.

(Note: A mostly accurate summary, not an exact transcription.)

So here I am, (sub-)module owner of the Add-ons Manager. It even says so on a wiki, so it must be true!

So what’s this mean? Mostly it means more work for me – and certainly new challenges, which is partly why I so readily said yes. I’ve been thinking more and more about direction, code quality, and solving problems in the “hard” to “impossible” range – but that’ll come in a later blog post. For now, it’s business as usual. And I’m in the business of kicking ass fixing bugs.

 

Solving Firefox’s add-on compatibility problem

“I want to upgrade Firefox, but my add-ons won’t be compatible.”

This makes me sad. And it’s a problem that’s been magnified by the switch to rapid release. One of the strengths of Firefox is it’s rich selection of add-ons. In fact, 85% of Firefox users have chosen to install an add-on. On average, those users have 5 add-ons installed. Firefox users really love their add-ons. So it’s not surprising that they get frustrated when one of their favorite add-ons gets disabled because it’s not marked as being compatible with the new Firefox update they just installed.

So I started working on a project to fix it. The end result will be that most add-ons will automatically be compatible with Firefox, starting with (hopefully) Firefox 10.

At the time of writing this, I’ve not yet finished the project. There are parts described below that I haven’t completed yet. But it’s progressed far enough to safely enable the new behavior on Nightly builds, starting with today’s Nightly build (labelled 2011-11-18).

Many of the changes are also on Aurora, but disabled. If you want to test the new behavior, go into about:config and change the extensions.strictCompatibility preference to false (true to re-enable the old compatibility behavior). Once I have a couple more major bugs fixed, we’ll make the call on whether to enable the new behavior by default on Aurora (which will ultimately become Firefox 10).

How does this work? The gritty details.

The problem with incompatible add-ons is that most of them are actually compatible – they just don’t know it. Unfortunately, blindly enabling all add-ons will enable some that are really are just incompatible, which will end in something breaking. So we needed a way to detect which add-ons would most likely be affected by incompatible changes to new versions of Firefox.

Assuming the extensions.strictCompatibility preference is set to false, add-ons will use the new compatible-by-default behavior. However, there are several reasons the Add-ons Manager will revert to using the old method of strict compatibility checking for a given add-on:

  • The add-on author chose to opt-in to strict compatibility checking by adding <em:strictCompatibility>true</em:strictCompatibility> to the add-on’s install.rdf
  • The add-on has been tested and determined to not be compatible with that version of Firefox (the Add-ons Manager gets this information from AMO)
  • The add-on uses a binary component
  • The add-on hasn’t been updated in an extremely long time (the compatibility data needs to state that it is at least compatible with Firefox 4, or Toolkit 2.0)
  • The add-on declares that is it only compatible with future versions of Firefox (we assume that add-ons are not backwards-compatible)

In the future, we also hope to give Firefox the ability to do a series self-tests to determine whether an add-on has broken something, and automatically mark that add-on as incompatible. Additionally, if the above heuristics end up producing false-positives, we may add the ability for AMO to tell the Add-ons Manager that it’s heuristics are wrong for a given add-on.

This also required a change to the way add-on updates are handled. Previously, users that disabled add-on compatibility checking didn’t get updated to the most recent version of their incompatible add-ons, since those updates often weren’t compatible with their version of Firefox (even thought they explicitly disabled compatibility checking). This meant that those users (which included most users running Nightly builds) didn’t automatically get important security updates of all their add-ons. The changes needed for the compatible-by-default behavior made it easy to support updating incompatible add-ons even for users that disabled compatibility checking, which means they’ll be running a more up-to-date and secure browser.

Finally, I should note that making add-ons compatible by default does not mean that add-on authors should stop including compatibility data, or stop updating it. Older versions of Firefox still rely on the compatibility data, and users can choose to opt-in to strict compatibility checking. Furthermore, add-ons need to still declare compatibility with at least Firefox 4 (or Toolkit 2.0).

 

—-
Gmail Archive Ukraine translation here – http://www.stoodio.org/archive-solving-firefoxs-add-on-compatibility-problem.

Status Update: AppTabs, Extension Manager UI

App Tabs and Home Tab

Status

  • Met with Limi to discuss design details, including:
    • BarTab-like functionality
    • Single-instance vs traditional tab-like behavior (preferring this)
    • Adding/removing AppTabs – dragging & context menu
    • Opening normal tabs when attempting a navigation action in an AppTab
    • Displaying less chrome when in an AppTab (and how awkward this will look for tabs-on-bottom)
  • Patch close to ready for foundation work
    • Had some fun with using yield for async tests
    • Learnt more about nsSessionStore than I wanted
    • Some debate over whether this implementation is the right approach (see bug [bug id=563730])

Loose ends

  • Need to resolve whether to keep current implementation approach or start over

Next steps

  • Finish and land foundation work in bug [bug id=563730]
  • Delve into adding some meat
    • Context menu
    • UI for adding AppTabs
    • Remembering added AppTabs
    • BarTab-like functionality
    • etc

Target for next week

  • Resolve implementation debate
  • Fix review comments, or start alternate implementation

Extension Manager UI Redesign

Status

  • Some reviews done
  • Mocking provider planning and implementation (not there yet)

Loose ends

  • None

Next steps

  • Mocking provider + tests
  • Fix top priority bugs

Target for next week

  • More work on mocking provider
  • P1 and P2 bugs as time permits, eg:
    • Bug [bug id=567127] – Add Install button for installing from local file
    • Bug [bug id=565682] – Support Drag & drop of XPI file
    • Bug [bug id=562890] – Add preferences button in list view

Miscellaneous

  • I’ll be in the Mountain View office during the week of 28th June

Reflections

  • I still get too stressed by certain things that are unavoidable when being a software developer

Status update: Extension Manager UI, App Tabs

Extension Manager UI Redesign

Status

  • Landed! Woo!
  • Backed out! Boo! Thanks to a Ts regression on OS X 10.5.8, which turned out to be bug [bug id=519893] again.
    • A little background: A specific OS function in OS X sometimes takes significantly longer if there is no monitor attached. So IT adds resistors to the monitor port of the Mac Minis that run Talos, to trick OS X into believing there is a monitor plugged in. Yes, this really is as ridiculous as it sounds.
  • Successful test day on April 30th (Henrik did an awesome job, over a 16-hour day)
    • Plenty of bugs found, but no show-stoppers. Some were known, others not. (And yes, finding bugs is a good thing.)
    • Also got a lot of feedback on the UI design
    • Test days mean a LOT of bugmail

Loose ends

  • Mossop will be re-landing soon

Next steps

  • Fix some annoying low-hanging fruit like bug [bug id=554237]
  • Implement manual updates pane (bug [bug id=562622])

Target for next week

  • This is temporarily low-priority for me, while I get stuck in on starting App Tabs (see below). So I’m not expecting to get a lot done here, sans any reviews that may come up from people submitting patches (hint hint).

App Tabs and Home Tab

Status

  • Picked up this project, together with Marco
    • Marco will be working on the Home Tab (a locally hosted home page that sits in a special App Tab)
    • I’ll be working on App Tabs (small persistent tabs that show a often-used specific site/page, and don’t get lost in all your other tabs)
  • Looked to see if any addon has this implemented in a way that could be uplifted, but didn’t find anything suitable
  • Started working on a proof-of-concept
    • Hit various roadblocks and discovered its going to be more work than originally anticipated
    • Tabbrowser makes certain assumptions everywhere that I need to burn with fire correct
    • Hoping that once the foundation is set, everything else can be separate bugs. eg: syncing between windows, make the documents single-instance, etc

Loose ends

  • None

Next steps

  • Finish foundation work in tabbrowser

Target for next week

  • Want to at least get the basic stuff working properly. Not sure whether that will include review-ready code or not (sadly, probably not)

Reflections

  • Regardless of how exciting an existing project is, starting a new project always seems somehow more exciting

Status update: Extension Manager UI

Extension Manager UI Redesign

Status

Loose ends

  • None

Next steps

  • Land it already, gah!
  • Fix the things we broke by landing it
  • Further stabilization and polish
  • Manual update process

Target for next week

  • Landing and damage control
  • High priority bugs – I’ll be re-triaging this week
  • Start on new manual update process

Reflections

  • Continually having to push back landing dates is frustrating
  • We still need to get more familiar with working on project branches
  • The scent of a new project really gets my mind racing

Status update: Extension Manager UI, Tab matches

This is a “I’m still alive” type of update. It’s been a couple of weeks since I last posted a status update. In that time, I’ve had a week off thanks to a 5-day weekend of public holidays (yay!), then being knocked down by a nasty flu bug (boo!). I’m back into hacking now but I’m still recovering, so things are progressing a bit slower than usual right now.

Extension Manager UI Redesign

Status

  • Missed the date we wanted to ideally get this landed on trunk (31st March) – this was disappointing, but there was just too much to get done (including reviews, as always). On the upside, we want it to bake in nightlies for awhile before being pushed out in an alpha.
  • Almost done the first round of reviews – feedback was mostly lots of small stuff, which is positive.
  • Lots of bugs fixed! Mostly a lot of little stuff. The QA guys are helping hugely.
  • Made it look prettier with icons from Boriss and some CSS work (NOTE: THIS IS NOT THE FINAL LOOK!)

Category iconsAddon item

Loose ends

  • None.

Next steps

  • Land on trunk.
  • Fix more bugs, and add planned features (including start of AMO integration).

Target for next week

  • Next round of reviews.
  • Fix remaining P1 bugs blocking landing.
  • Assuming this all goes smoothly and the API is ready to too, we can land it on trunk (!!)

Tab matches in Awesomebar

Status

Next steps

Miscellaneous

Reflections

  • I should know better than to overwork myself and get burnt out.
  • Books are awesome.
  • Hackers around the world are selling their soul for a certain new device that is the start of a computing future they do not want. This saddens me.

Status update: Extension Manager UI Redesign

Extension Manager UI Redesign

Status

  • Install/uninstall/update working really nicely. This took longer than expected, due to:
    • Some issues with the new API – Mossop has added various new functions, properties, and events
    • Some bugs with the API that tests didn’t find – these are being fixed by Mossop, with additional tests added
    • Various things I haven’t considered or anticipated (such as providers installing addons without using an AddonInstall object)
    • I had some preconceptions about the flow of events and state of installs/upgrades which turned out not to be entirely accurate
  • Various CSS and layout tweaks (such as making buttons look pretty) – CSS3 is my friend
  • Various bugs I found along the way
  • Filed a bajilion bugs on the UI and API – things that need fixed, features that need implemented

Loose ends

  • Waiting on some placeholder icons from Boriss

Next steps

  • More bug fixes!
  • Reviews
  • Initial landing on mozilla-central
  • Write a mocking provider, for automated testing (and additional dogfooding of the private API)

Target for next week

  • Add support for addon installs that don’t use an AddonInstall object and its events (bug [bug id=553515])
  • Make enable/disable prompt for restart (bug [bug id=553631])
  • Fix display when no update is found (bug [bug id=553870])
  • Consolidate “Restart Now” buttons to just one entry (bug [bug id=553460])
  • Any other P1 and P2 bugs I can get to
  • Start writing mocking provider
  • Initial reviews

Reflections

  • Planning and testing are necessary but not adequate on their own. Dogfooding will invariably find issues that planning and testing could not.
  • Spare time suffers from Parkinson’s Law (and at times, both of them)

Status update: Firefox team work week

Tonight, on a very special status update, I’ll recount the recent Firefox team work week.

The Firefox team work weeks are pretty special. For one week, every 4-6 months, the whole team gets together at the main Mozilla office in Mountain View. Like the rest of Mozilla, the Firefox team is spread around the globe – around half the team works from various other offices (such as the Toronto office) or from our homes. For some of us (like me), getting to Mountain View can mean a 12 hour international flight – with all the jetlag that accompanies it. Being so geographically dispersed, it’s important we get together every now and then. There are some things that still work best when everyone is face-to-face.

Getting there

I had booked my flight to arrive in California on the Saturday before the work week. I’d learned my lesson previously about taking time to recover and adjust, after spending over 12 hours in a flying sardine can. I don’t sleep well on planes, so I slept at the hotel most of Saturday afternoon. We stayed at the Hotel Avante, which is relatively close to the office (25min walk) and provides Wi-Fi (even if it’s a bit slow). This trip I discovered the Avante has a pool out the back – I didn’t go in, but it was nice to lounge out there (alone!) in the sun with my laptop and catch up on email and some work.

The Avante's poolMe by the pool

Monday

Early Monday morning was a shock to the system, as all mornings are. Working from home in an odd timezone means I usually get to start work at a leisurely hour, and work late. When I’m in Mountain View, my alarm is set for 7am. This seems to amplify jetlag by an order of magnitude.

Those of us staying at the hotel car-pooled to the office, and the day started with strategic planning. And by “strategic planning” I mean we wandered around confused in the newly-opened 2nd floor office space, as none of us staying at the hotel had seen it before. We eventually found our conference room, and scavenged some tables. They were arranged in what I like to call a “square round table”:

Square round tableSquare round table

After some general catching up and setting up, we all got the chance to attend the weekly Mozilla project meeting in person – rather than having to dial in or watch a live stream. After quickly grabbing lunch, we had the usual weekly Firefox team meeting – again, in person! This was mostly planning the week ahead of us; lovingly called “agenda bashing”. Rather than having every hour scheduled for big meetings as in previous work weeks, we had a lot more free time for hacking and, of course, scheduling smaller meetings. Monday and Friday were officially “travel days”. This meant that for those of us around on these days, it was mostly free time for hacking and meeting with specific people. Wednesday was also specifically blocked off for this.

Tuesday

Tuesday started with a round-table discussion with Mike Shaver, Mozilla’s VP of Engineering. Shaver is great at rallying the troops, and talked about productivity, working smarter, and generally getting stuff done (amongst other things). This was followed by a feature roadmap discussion. These discussions were pretty high-level (read: abstract), without much technical detail. It was these types of discussion that are best done face-to-face, and it was really worth it.

After lunch and some hacking time, the QA team joined us (they were having their work week too). We discussed ways of further involving QA in feature development, getting more relevant information to QA during development, and generally how to better help each other. There was also some discussion on project branches, which we’re using more and more. A lot of my work has been with larger projects and project branches, so I was able to answer various questions QA had, as well as explain how that development process differs compared to bug-sized projects.

The Jetpack team then joined us, to present the new Jetpack SDK. They discussed the SDK internals, Jetpack development, how it differs from the original Jetpack experiment, and the eventual integration into Firefox. This was followed by some Jetpack hacking – some working on Jetpacks, some on the SDK’s APIs.

At this point, jetlag hit me like a wrecking ball.

Wednesday

Wednesday was an all-day hacking and free-for-all session, with plenty of discussions with awesome people – some of these were spontaneous, some not. These ranged from UX and API discussions about the new Extension Manager, to showing off my project status dashboard and my ideas for it. Before lunch, I sat down to listen to Gen Kanai talk about Firefox in Asia. Afterward, I had a planned meeting with everyone involved in the Extension Manager rewrite – with people from the Firefox, UX, QA, and AMO teams. This was primarily to discuss the QA plan for that project – test plans, development process, milestones, schedules, etc. Later in the afternoon, I caught up with Aza Raskin – he had a cool demo to show me.

In the evening, there was a rather unusual event planned for us. We traveled to Whole Foods in Cupertino, where 2 chefs and a kitchen were waiting for us. With guidance from the chefs, we were split into 4 groups – each group responsible for cooking one course of a four-course meal for the whole team. It was a really fun time, and the food we cooked up was pretty good!

Cooking classCooking class

Thursday

Thursday involved a lot of free hacking time for me, since I didn’t need to be at various scheduled meetings. During lunch, Jinghua Zhang presented the results of the recent Test Pilot study on Firefox menu usage (sans pretty graphs). This answered a lot of questions we had about Firefox menus, but raised some new questions too. I found it particularly interesting that people have difficulty finding the “Check for updates” and “Add-ons” menu items. Someone brought up the possibility of regularly doing this study, which seemed like a great idea.

Later on, we had the weekly Extension Manager redesign meeting with the AMO team. Considering we’d had such a comprehensive meeting the previous day with QA, we still managed to cover quite a lot. We discussed various AMO integration ideas (such as the “Discover” pane), design direction, helping users learn about add-ons, privacy, security, and the new Rock Your Firefox.

Friday

Friday was a travel day, with most of the out-of-town people flying out early in the day. My flight was much later, meaning I didn’t need to leave the office until mid-afternoon. So I lounged in/on two over-sized beanbags, surrounded by the remainder of the team, who sat at desks. It always feels weird leaving the office at the end of the last day, knowing I won’t see any of my co-workers in person again for a number of months.

Reflections

  • OMG jetlag – WHY?!
  • Walking down the street carrying 10 large pizzas gets people’s attention.
  • MV people tried to convince me to move to MV, while the Toronto people tried to convince me to move to Toronto. This felt awesome, but I’m still not moving in the near future.
  • I work with very smart people.
  • Some problems are best solved face-to-face. Generally, these are not technical problems.
  • Switching from working remotely to suddenly working in a busy office for a week is difficult to adjust to (and stay productive). But doing this periodically is important, as it helps you understand the team better and put things into perspective.