Tab matches in Awesomebar

Status

• Finished nsPlacesAutocomplete integration - works wonderfully well
• Filed bug 530209 to change the preferences UI to allow adding tab matches
• Tryserver builds at http://people.mozilla.org/~bmcbride/tabmatches/latest/

Loose ends

• Waiting on feedback

Next steps

• Unit tests
• Respond to feedback

Target for next week

• Unit tests

Binding for untrusted text in security dialogs

No change.

Miscellaneous

• Helped with some lightweight theme bugs for 3.6

Reflections

• Sometimes, there is no good solution. But there is a best solution.

Related posts:

1. Status update
2. Status update
3. Status update

And I did this on purpose.

My apologies to the people who were confused, bewildered, and frustrated by this. You were all (including co-workers!) unwitting participants in my experiment. It showed that the current interface for Taskfox features may not be discoverable enough, even for technologically savvy people. Thankfully, we think we have a solution - more on that in a future blog post.

In addition to the lack of information, the Map task was broken. This wasn't part of the experiment - merely a mistake on my part. Same with the inability to enter spaces (thanks to a last-minute change). Oops.

So today we have a new build! Bugs fixed, some things added, and some information on how to get the most out of using Taskfox.

Using Taskfox

When you open a Taskfox build, its the same as any nightly Firefox/Minefield. It displays web pages the same, the toolbars and menus are the same, the URL bar is (almost) the same. But typing in the URL bar is where the magic begins, as you can see in the following screencast:

View on Vimeo.

Where to provide feedback

There are plenty of ways you have give us feedback:

• Comment on this blog post
• Post to the dev-apps-firefox mailing list (also on Google Groups)
• Join #fx-team on IRC
• Join the weekly meeting every Thursday at 3:00pm PDT (10:00pm UTC)
• E-mail me directly (although I'd prefer one of the other ways)

Getting Taskfox

Builds are available for all platforms that Firefox ships on:

• Windows (or without the installer)
  
• OS X
• Linux

Related posts:

1. Taskfox – guineapigs wanted
2. Ubiquity & Firefox – Introducing Taskfox

We've been working steadily on the project since then, and a lot of progress has been made - to the point where Taskfox is both usable and useful. Which also means the project is at the stage where its very important and beneficial to have a lot more people run it daily - using it, testing it, and providing feedback. That's where you come in.

There's links to the latest builds at the end of this post, if you want to just jump right in to playing with it. But I'd like to go over a few things first.

Development Status

Since Taskfox is still in development, there are only a limited number of tasks that are available for now. As of today, these are:

• Amazon
• Google
• Images
• Map
• Music
• Netflix
• Translate
• Video
• Weather
• Wikipedia
• Yahoo

Additionally, your installed search engines will be automatically available as tasks. A list of other potential tasks is here: https://wiki.mozilla.org/Taskfox/Verbs

There's also a few features that simply aren't implemented yet:

• Simple keyboard navigation
• Localization of tasks
• Showing the task in the URL bar when visiting a related web page
• Task management
• ???

Thoughts and Feedback

I'd like to go over what thoughts and feedback would be most useful for us right now.  Of course, any and all feedback is welcomed and appreciated - whether its big or small. But there are a few things we'd specifically like to get answers to for this round.

As you may recall, Taskfox has three basic principles:

• Taskfox must help users accomplish tasks
• Taskfox must be useful to all Firefox users
• Taskfox must be fast, secure, and extensible

We want to make sure that Taskfox sticks to these principles.

Beyond that, we want to know how it feels:

• Does it feel like Firefox?
• Does starting a task feel right?
• How does the process of performing a task feel?
• How does it meld with your workflow?

And finally, we want to know what you would change to make Taskfox even more useful.

• What tasks do you do regularly, that you wish could be done in one simple step?
• What tasks do you wish you could do in your browser, but are too complicated?

Where to provide feedback

There are plenty of ways you have give us feedback:

• Comment on this blog post
• Post to the dev-apps-firefox mailing list (also on Google Groups)
• Join #fx-team on IRC
• Join the weekly meeting every Thursday at 3:00pm PDT (10:00pm UTC)
• E-mail me directly (although I'd prefer one of the other ways)

Getting Taskfox

Now for the goodies I promised. I'll be generating builds regularly - some may have major changes, while others simply bug fixes. These builds are based off Minefield - a developmental branch of Firefox that contains the newest experimental code. While I try my best to make sure things aren't broken, there are no guarantees.

Note that these build's won't (yet) automatically update. You'll need to manually update them periodically. To make that easier, I'll be updating the Latest Build link on the wiki, posting information on the dev-apps-firefox mailing list (also available through Google Groups), and posting to my blog (which is aggregated to Planet Mozilla).

I should also point out that since these are development builds, they're only available in the en-US locale for now.

Builds are available for all platforms that Firefox ships on:

• Windows (or without the installer)
  
• OS X
• Linux

Happy testing!

Related posts:

1. Taskfox, the screencast edition
2. Ubiquity & Firefox – Introducing Taskfox
3. Status: Jetpack and Content Scripts in Firefox

At the same time, it's clear that Ubiquity is not for everyone. Ubiquity users are people who:

• are willing (and brave enough) to experiment with alternative, natural language keyboard style interfaces
• use web applications extensively for email, calendaring, document editing, online collaboration and most of their day to day tasks
• understand that the web is a collection of data which can be remixed, mashed together, and edited by users as well as by web developers
• are familiar with Firefox Add-Ons and willing to install Ubiquity on all their machines
• know at least some basic English

However, the underlying benefits that Ubiquity offers are universal. All internet users perform a given set of tasks regularly - and many of these tasks require the same set of steps to be taken every time. The browser should facilitate accomplishing the end-goal and help reduce the number of steps to reach that goal whenever possible.

That's where Taskfox comes in. Taskfox is the codename of a project to uplift parts of Ubiquity into a future version of Firefox (ie, after Firefox 3.5 ships). The basic idea behind Taskfox is simple: take the time-saving ideas behind Ubiquity, and put them into Firefox. That means allowing users to quickly access information and perform tasks that would normally take several steps to complete. This does not mean "ship Ubiquity with Firefox". While the two projects share some common ideas and use cases, the goals are not all the same. What would this look like? We don't really know at the moment, though we have some ideas.

When thinking about Taskfox, we've been trying to keep in mind three basic principles:

• Taskfox must help users accomplish tasks
• Taskfox must be useful to all Firefox users
• Taskfox must be fast, secure, and extensible

Accomplishing tasks

The workflow that people currently have while using Firefox is very important. Interrupting that workflow by being too "helpful" would be very disruptive, as would forcing a completely new workflow. Unfortunately, this is something that Ubiquity does - in order to use it, you must adopt a new workflow. Taskfox needs to be available within your existing workflow - working with it, not against it.

In addition to being part of your workflow, Taskfox also needs to be obviously reachable. One of the issues many people have with Ubiquity is figuring out where to start typing. By using a method people already use to perform tasks, we can get this connection. Zac Lym's usability studies found people expecting to be able to type into the location bar. This makes sense, given that location bar is by far the most used area for performing the most universal action (going to a website).

Available and useful to all Firefox users

One of the things that Ubiquity is known for is its use of natural language for input. Firefox ships in more than 70 languages. These two things do not easily mix.

So let me be blunt here: natural language input is NOT a goal of Taskfox. It never has been, and it never will be. Nor is it intended to bring the user and the web together via language.

However, Taskfox will be localizable. All locales are first-class citizens here. This will include both string localizations and locale-specific content providers (just like there are locale-specific default search engines). This also implies that Taskfox will include far fewer commands by default than Ubiquity. That's OK, though, as we want to pick and choose the ones that are felt to be most useful to the widest audience.

Speed, Security, and Extensibility

In order for Taskfox to succeed, it needs to not regress performance. As we develop the project, we will ensure that we do not have a negative impact on any of the key performance metrics by which Firefox is judged.

Also, Taskfox will not include the ability to subscribe to Ubiquity-style command feeds, as this has the some potentially nasty security implications. Similarly, its not a playground for experimental UIs - it needs to be usable for power users and grandmas alike (and even power user grandmas), not just the early adopters.

However, there will be an API for extension developers to use. This will be available to normal Firefox extensions - and so any such extensions will go through the same security and quality checks that any other extension on AMO would go through.

Progress

Right now, it's mostly me, with some help from Atul and Jono and early user interface thinking from Alex and Aza. I'm working on migrating code from a proof-of-concept extension to a new Incubator repository at http://hg.mozilla.org/incubator/taskfox, which contains a clone of mozilla-central (ie, the bleeding edge of Firefox code). As the project progresses, we expect to occasionally merge into mozilla-central and be available in Firefox nightlies. Until then, you'll have to build Taskfox on your own, though we'll also occasionally post some experimental builds for people to try out.

Also, we've set up a Taskfox page on the Mozilla Wiki to document everything about this project.

Starting this week, there will be public meetings held every Thursday at 3:00pm PDT (10:00pm UTC). You can find when that is in your timezone here. These are open to everyone, no RSVP needed. To join in, dial in to one of the following numbers:

• +1 650 903 0800 x92 Conf# 8604 (US/International)
• +1 416 848 3114 x92 Conf# 8604 (Canada)
• +1 800 707 2533 (pin 369) Conf# 8604 (US Toll Free & Skype)

You can also join #fx-team to chat about Taskfox on IRC, or discuss it on the mozilla.dev.apps.firefox newsgroup. We welcome anyone to help out, file bugs, write patches... get involved!

Related posts:

1. Taskfox – guineapigs wanted
2. Taskfox, the screencast edition
3. An Introduction

And this isn't just a coding problem: it also directly affects the user. Not only are there several methods of asking the user for information, but the information is fragmented in the system, so its not uncommon to need to fill in the same information more than once.

Lets tackle the user-facing issues first, in true Ubiquity fashion. This will be Part 1 in a mini-series of blog posts on this topic. Part 2 will focus on the developer's side of the issue. So lets get started.

Data vs Information

One part of this problem is that some commands want data entered, some request information, while others use knowledge. You can read about the differences between these concepts here. To summarize and bring that into context: data is what the code deals with, and it should stay there exclusively. People don't think in data, but in higher levels such as information, knowledge, and wisdom. Thinking in data is difficult - which is in contrast to a computer, where data is the native language and using higher-level constructs is more difficult.

And so we have to choose - do we make ourselves work harder, or do we make our computers work harder? Hopefully the obvious answer is the later option.

A simple example of this difference is entering a date/time. Ubiquity uses the awesome date.js library for this. It allows you to say "next Thursday", as opposed to figuring out what date next Thursday is, and then figuring out what format is expected. Neither of those tasks are trivial. Sure, you may be used to doing it and maybe even be able to do it quickly, but its still a lot more work than just "next Thursday".

Fragmentation

Unfortunately, its not uncommon to be asked to enter the same information more than once. This isn't just a problem with Ubiquity, but with computers in general. Since information/preferences/data is stored in various different methods/locations/formats, its next to impossible to consolidate it system-wide. But the problem can at least be made to not get worse within (at the very least) Ubiquity.

Part of this can be solved by going back to the data vs information problem. When data is stored, it can take on many different meanings, since its meaning is dependent on the code that uses it. But if information is stored, then although the use may vary, the meaning behind it is global.

As an example, lets look at what's required to login to a web service. The user generally has to supply at least 2 pieces of information: a login name, and a password. But where's it stored? In the password manager? In the preferences? Which URL should be used as a key? The front-page URL? The login-page URL? The mobile version's URL? The web service API's URL? What about automatic logins, so the user doesn't have to confirm every time - if the user has more than one login for that web page, how is it decided which one should be used?

A lot of the above issues (if not all of them) can be solved by reducing fragmentation, and storing information and knowledge rather than data. Picture this: Instead of associating login information with a specific URL, its associated with a uniquely identifiable name of the web service. Then each URL that is trusted to be associated with that web service is associated with the web service's name, and this association is automatically discoverable (and verifiable) without user input. Finally, all of this information is stored in such a way that its just as easily accessible to the browser, extensions, Ubiquity commands, local applications, and the user - on all of their computing devices.

Now instead of storing fragmented data, we have consolidated information and knowledge.

A solution

To finish off Part 1, I'm going to describe the user-facing component of a solution. Part 2 will cover the technical details, including an API.

If you've used a Ubiquity command that requires user input or some form of configuration, you would have no doubt noticed that the command has to provide its own method of presenting this task to the user. This invariably leads to multiple "setup" type commands - one for each command that needs user configuration. On top of that, these setup commands themselves are inconsistent - different naming schemes, different way of entering the same types of information. For instance, do you use separate setup commands for your login name and password, or enter them both in the same command? If its the same command, what happens when you change your password? I think the only thing these setup commands have in common is that they're annoying to use.

But what if all you needed to do was tell Ubiquity "my twitter login is theunfocused". Or "my home is Dunedin, New Zealand". Or "my google apps domain is blairsawesomewebsite.co.nz". Simple, yes? But what if you want different information to be used for different commands? Say you wish to use the "mozillaubiquity" twitter account when using the "twitter-timeline" command. Simple: "my twitter login for twitter-timeline is mozillaubiquity".

That's it for Part 1. As mentioned earlier, Part 2 will focus on the technical details and an API. Comments are not only welcomed, but wanted!

Related posts:

1. How web services punish you
2. Mozilla and Me
3. Ubiquity & Firefox – Introducing Taskfox

I'll be working from home, in the beautiful city of Dunedin, New Zealand.

Here's a quick-n-dirty list of what I'll be working on, in no specific order:

• Catching up (after buying a house, then going on holiday, I'm a bit behind)
• Ubiquity architectural changes
• Ubiquity metrics
• Development related Ubiquity commands (not just Ubiquity-specific!)
• Take over the world
• Integrating Ubiquity into the Firefox awesomebar
• Improving Firefox's Library window

And of course, that's just for starters. I've got plenty of ideas I haven't had time to flesh-out, let alone start implementing - ideas relating to not only Ubiquity and Firefox, but other Mozilla-related projects as well. So I've got plenty to keep me busy for a very long time!

On top of all that, I'll also be blogging more on what I'm working on. Not just to share my progress, but also to gather some all-important feedback from the community. I'm also going to try to use this newfangled thing called Twitter. I may have written the original Ubiquity command for Twitter, but I've never actually used the service myself. I guess this is a good of a time as any to start. And as always, I'll be on IRC - my IRC nick is Unfocused.

Related posts:

1. Ubiquitous information – Part 1
2. An Introduction

Once you've subscribed to that command, all you need to do is open Ubiquity, and start typing "private-browsing". Typing "p" may even be enough, depending on the commands you have. By default, it will toggle Private Browsing to the opposite of the state its currently in. You can also explicitly specify whether you want it on or off by specifying "on" or "off" respectively. That way you don't have to actually remember or check whether you're already in Private Browsing.

Screenshot of Private Browsing command

The command can be subscribed to from here:
http://theunfocused.net/moz/ubiquity/verbs/?cmd=private-browsing

Note: You'll need to be running a nightly build of Firefox 3.1, or 3.1 beta 2 (when it gets released) or newer.

Related posts:

1. A good and safe Internet
2. The New House
3. Ubiquity & Firefox – Introducing Taskfox

In other words, it punishes the user because the web service doesn't trust the 3rd party application. This is wrong. Its not the user's fault.

This has come up many times for Ubiquity command authors trying to provide easy and novel access to web services. It means that if commands want to use a web service's API, they need a setup procedure before they can be used. Which is stupid, because a command already has access to the login cookie, and the login information in the password manager.

I'm not saying that I'm providing a solution for all of this. There are already plenty of other existing methods to choose from - some of which may fit, some of which may be just as bad. What I am saying is that while granularity in access control can be seen as a good thing, it should not come at the user's expense.

Related posts:

1. Ubiquitous information – Part 1

Firstly, a short introduction about myself and what I'm doing. My name is Blair McBride, and I live in Dunedin, New Zealand. I'm enrolled at the University of Otago and I'm interested in amateur photography, programming, and making things easier to use. At Otago, I have completed an undergraduate degree in Computer Science, and continue to learn and work with computers outside of academia. Now I'm working on completing an undergraduate degree in Psychology, while also exploring other areas of study such as Art History, Latin, and Anthropology/Archeology. Computer Science and Psychology may seem like an unusual mixture, but I believe its important to understand how the brain works so that computer interfaces (and physical objects) can be designed for the people using them, rather than designed for computers/manufacturers.

This leads me to a topic which is likely to be featured a lot in my blog posts: Ubiquity. Ubiquity is a project from Mozilla Labs, the virtual lab responsible for various experimental browser addons such as Weave and Snowl. The principle aim of Ubiquity is to experiment with connecting the Web with language. In doing so, we're attempting to find new user interfaces that could make it possible for everyone to do common Web tasks more quickly and easily. Instead of being forced to do things the way the computer thinks, it allows you to tell your computer what to do in the way that you understand it. In other words, it lets you control the computer, rather than having the computer control you. We recently reached the stage where Ubiquity could be useful for normal people, and so the first public release was unleashed upon the world. In conjunction with the official launch, Aza Raskin posted an introduction to Ubiquity and an more in-depth followup, each of which covers the basic concepts and usage. But as he explains, Ubiquity is more than just an interface - its also a development platform that lowers the bar for entry into augmenting the user's browsing experience. In plain English, that means Ubiquity makes making Firefox addons easy. Really easy. Just be aware that Ubiquity is still in its early stages of development. There are bugs. There are things that will change, some drastically. But all for the better, because Ubiquity is going to change the way people interact with their computers.

I started contributing to the Ubiquity project sometime after the basic foundations had been laid. But since then, I've been able to find bugs, fix bugs, add features, contribute ideas, and meet some great people. And the development process is made as transparent as possible, so everything is public and anyone can contribute.

Related posts:

1. Ubiquitous information – Part 1
2. 7 Unsolicited Things
3. How web services punish you