Is there a quantitative measure for usability?

Someone asked me the other day if there was a quantitative measure for usability. Me, being a person who doesn’t lack an opinion, laid out my philosophy. Thought it would be useful to post this to my blog as well, in case anyone else was interested in my answer.

Usability is measurable. If tested, usability needs to be measurable. It’s about task completion, steps to success, faults in the process, number of clicks/taps to get to success. We can take a design, assess its usability, take a competing design solving same task, assess its usability, and compare. A is more usable than B. It’s pure metrics and HCI.

Where things get tricky is if we talk about measuring user experience. That is not measurable because user experience is through the lens of the person using it and leans heavily on variables like history, perception, and emotion (to name a few). Sure, at the end of a usability test we can ask how using an object or interacting with a website made someone feel and compare metrics, but that doesn’t tell the whole story about the user experience. User experience is based on the current frame of a person and the environment she is in.

Let’s use an example to illustrate.

Let’s say I sit you down in a controlled lab environment and ask you to diagnose and solve problems on your Xbox. From a usability standpoint, I can measure two or more approaches and definitively say one is better than another given a statistically significant population of test subjects. From a user experience standpoint, I can ask how satisfied someone is and make a judgment about what is the best user experience, even if the measurement is inherently flawed.

now …

Let’s say you sit down at home with a broken Xbox while an online Halo tournament you want to play in for money is happening right this second and you are forced to diagnose and solve problems on your Xbox. Your frustration and impatience is magnified. The process of self-diagnosis will quickly go off the rails and turn into a bad user experience for you. This isn’t a usability or user experience issue we can solve; We are dealing with a person with a frame-of-mind that can never truly can’t be satisfied. The “test” is rigged. Unless we can somehow concoct a solution that relies heavily on magic, making your red ring of death go away all on its own, there’s little chance the Xbox diagnosis experience will come out of this situation smelling like roses.

So even though we can “measure” user experience in a lab environment after a usability test, it’s my contention that the measurement is nothing more than hogwash.

As my good friend Matt Conway points out, usability is boring. (Gasp!) It is for the worlds where millions of dollars or someone’s life is on the line when things like error rates, task completion, and static analysis matter – think medical equipment, jet cockpit interfaces, large-scale enterprise computing. User experience is about making something magical or delightful. There is no measure for that, even in aggregate.

Measuring stuff right isn’t as important as measuring the right stuff. And the right stuff is really hard to measure when we’re talking about user experience.

Leave a Reply

Your email address will not be published.


three + 8 =