Can I use it? Evaluating Usability through Quick-UX.

user-useit The first of the 3 primary components of Quick-UX, of which I will be discussing in greater depth, is the one of Usability. Put simply, Usability is a measure of how easy something is to use.

In sticking with the primary goals of Quick-UX (quick assessment for summary, directional guidance, and quantitative comparison) the variables constituting the minimal representative subset for Usability are…

  • Accessibility,
  • Consistency,
  • Recognition,
  • Navigation, and
  • Page Load Time.


Accessibility is the measure of how many differently skilled/abled types of people (including individuals with disabilities) in varying locations (e.g. mobile web) can make use of a given product. There exist many, very thorough, guidelines for determining the degree to which a product adheres to accepted accessibility standards. However, many can be very complex and time-consuming, also requiring the study of a good deal of the underlying code — much of which goes against the goals of the ‘quick’ part of Quick-UX.

I use a robust (and free) proxy for quickly assessing a product’s Accessibility through the use of the Functional Accessibility Evaluator (fae) link. The fae’s resultant scores are averages which, in turn, are normalized to a range from zero to one to represent the value for Quick-UX‘s Accessibility variable.


Consistency is a fundamental component of Usability. The less learning a new user has to do to use a product the more usable is that product. Products should not have multiple interface elements, page layouts, or content that are used for the same purposes, but vary depending on how the user got their or where they are currently looking.

For example, if the product/site is in the travel industry and the site often references ‘travel search engines,’ a consistency that can grow confusing (inconsistent) is when the language that describes the same engines varies from instance to instance, from ‘engines’ to ‘tse’s,’ to ‘search travel engines,’ etc.

The determining of the value for the Consistency variable is done through the brief surveying of the product, and assigning a…

  • 1 if there are no apparent inconsistencies,
  • 0.5 if only minor, non-intrusive inconsistencies are found,
  • 0 if there exist inconsistencies on major element(s) or a majority of minor elements. Inconsistencies on major elements lead to immediate confusion and second guessing information being conveyed.


The measure of the Recognition and intuitiveness of a product conveys how easily an average user of a product can immediately grasp how to use it. When evaluating this Usability variable remember… YOU are NOT an AVERAGE user. The Recognition variable is assessed from the perspective of an average user and is assigned a value of…

  • 1 if the interface and product, in general, feels familiar and is easy to use,
  • 0.5 if some poking, finesse, and interaction are required before the user will be able to gather his or her bearings in the use of the product,
  • 0 if the average user will have clear difficulty understanding (1) how to use the product and (2) what the product is trying to communicate.


Evaluating the Navigation variable as it relates to Usability (and Quick-UX) also includes, in addition to site navigation, the review of the site’s flow, transitions, interactivity, and clear communication of progress. If a user can’t figure out how to get from point A to point B, or is not presented with clear information as to how he or she got to point C or that there remain points D through Z to still travel, the overall Usability of the product can be sorely damaged.

The Navigation variable is assigned the value of…

  • 1 if the product presents a straightforward decision process, leveraging animated transitions when appropriate, providing clear feedback, and communicating progress within each multi-stage task,
  • 0.5 if the two following conditions are met:
  1. occasional, but easily correctable, mis-steps in accomplishing tasks and/or completing processes occur, and
  2. there exists a visible current progress indicator for all multi-step tasks,
  • 0 if any of the following scenarios occurs with frequency:
    • resultant Interaction or other resultant event occurs contrary to the desired decision path,
    • surprised by result of interaction, or
    • no communication of progress, flow, or navigation.

Page Load Time

There are some products out there (Twitter comes to mind) that could not possibly have an easier to use interface coupled with a simpler purpose (to say what you are doing) that are frequently rendered barely, or completely, unusable due to entirely unacceptable product responsiveness.

A company can have the best product around, but if the pages are too sluggish, they can achieve a real pain-point in the overall user experience, rendering a product unusable.

Assessing the Page Load Time variable requires very little of your time. However, I do recommend you average at least a few data points over the course of a day, or days, to make sure you have an accurate sense of the normal product responsiveness.

  • If the product typically loads the information promptly (within acceptable expectations) then the Page Load Time variable is assigned the value of 1.
  • If the product exhibits the occasional, inconsistent delay, use 0.5.
  • And, if the product (like Twitter, at the current moment) has frequent and long delays (including outages) the value for Page Load Time variable is 0.

Usability. Quickly. Done.

The quantitative assessment of these variables are structured to provide a quick and painless method of evaluation to form a summary, directional guidance, and/or values that facilitate inter-product comparisons through the answering of the basic question…

Can I use it?

When answers to the above question of Usability is combined with…

Should I use it? (Usefulness)

Do I want it? (Desirability)

… the result is a product’s overall, repeatable, quantitative assessment of User eXperience.

Enjoy! (and discuss)

Jeremy Horn
The Product Guy


Quick-UX. Quick Heuristics for User eXperience.

heuristic Introducing… Quick-UX

There are many procedures, rubrics, methods, analyses and theories with respect to assessing the overall User eXperience of a product. Most User eXperience assessment approaches require a heavy investment of time, analysis and effort.

Often, a quick assessment of User eXperience is more aptly called for. A quick assessment allows for rapid compilation of simple heuristics that can be very handy in providing…

  • a summarized view of a product’s overall User eXperience,
  • directional guidance for a product’s future development, and/or
  • metrics for comparison with other products.

More detailed and extensive heuristics (many with hundreds of variables to evaluate) exist that delve into the finer details that lead to more directly actionable steps, but they are neither cheap in time nor money.

Quick-UX is a method that I have developed and refined over time and frequently utilize when quick assessments are best suited to the task. The method that I describe below is a great way to build a summary description with quantifiable and comparable metrics, representing the understanding of the overall User eXperience of a product.

The Quick-UX evaluates the degree to which a product successfully addresses the following 3 questions:

The elements evaluated in response to each question constitute a minimal representative subset that accurately addresses the question posed while adhering to the goals of Quick-UX.

Evaluating Usability

Whether or not something can actually be used is critical to a product. Quickly evaluating the ease with which the typical consumer can use a product consists of looking at the following variables:

  • Accessibility,
  • Consistency,
  • Recognition (also including Intuitiveness),
  • Navigation (also including site-flow, transitions, and interactions), and
  • Page Load Time.

Each variable here, as well as those making up the other 2 evaluation categories (‘questions’), possess specific rubrics to generate repeatable and quantifiable values (each normalized to be of the range from 0 through 1 or 2) that can consistency be understood and compared to like evaluations. For the Quick-UX, each category can, through the summing-up of each of its values, achieve a maximum category score of 5 (minimum being 0).

Evaluating Usefulness

What good is a product if it isn’t useful? Does the product solve a new problem, or an existing problem? Does it do so in an innovative and creative way? The quick evaluation of Usefulness is constituted by an assessment of these variables:

  • Functional Expectations (expectations are created via marketing, content, branding, etc.; put another way, ‘were the functional expectations of the product, from the perspective of the consumer, achieved?’),
  • Errors (including handling, recovery, and prevention),
  • Product Differentiation (including memorability),
  • Findability (i.e. search engine friendliness), and
  • Credibility.

Evaluating Desirability

The Desirability of the product, its appearance and the feelings it can stir in the user through the methods of presentation, can often be a good proxy for the organic word-of-mouth campaigns and buzz that spring up around it. Desirability taps into the emotions of the product’s users through:

  • Aesthetics (and Minimal Design),
  • Page Layout,
  • Color Scheme (including Contrast), and
  • Typography.

Desirability consists of only 4 variables. Aesthetics, unlike all of the other variables in the Quick-UX, is normalized to a range from 0 through 2, due to its greater impact on the overall desirability of a given product.

Try it. Use it. Tune it.

All of the values, when summed up (max. value of 15), form the UX (User eXperience) Rating for the evaluated product. The higher the rating, the better the product’s overall User eXperience. My favorite use for this evaluation is to quickly compare the User eXperience of multiple products with one another.

As the Internet and online products evolve I update and adapt my approach to quick evaluations. Let me know how the Quick-UX works for you. Tweak it, adjust it to suit your particular goals. Please share your findings.

What sort of quick User eXperience evaluation methods do you use?

More Information

You can read more about alternate User eXperience evaluation heuristics and theories at these websites…

  • Ten Usability Heuristics (link)
  • Stanford Guidelines for Web Credibility (link)
  • The User Experience Wheel (link)
  • Metrics for Heuristics: Quantifying User Experience (part 1 part 2)
  • User Experience Design (link)
  • User Experience Strategy (link)
  • How To Quantify the User Experience (link)
  • Designing Interfaces (link)

Enjoy & Post experiences with Quick-UX and other methodologies.

Jeremy Horn
The Product Guy

Simplify. Simplify. Simplify.

00_logo …So echoed Darrell Silver, one of the four co-founders of CommandShift3, as he explained one of the guiding mantras of and with whom I had the great pleasure of being able to sit down and speak with recently at his cool downtown digs. We discussed a range of topics, from the ideas and processes that led to the CommandShift3 web site to what features its community of users can expect to see as the product evolves.

CommandShift3 had 1mm page views within the first month after launching. A large factor in achieving this number most definitely comes from their keen attention to the User eXperience.

How did CommandShift3 zero in on such a rockin’ User eXperience?

Darrell best described their product experience design process as…

“iterating through to simplicity”

When the CommandShift3 group approached designing the User eXperience, simplicity was always in the forefront of their thinking. Every decision revolved around this guiding rule, from the potential use of tabs to where and if to use AJAX.

The Screenshot

One of the early challenges faced, one that can almost be a metaphor for the iterate-to-simplicity process, was with respect to the display of the websites that participate in the voting.

How do you size the screenshots correctly?
How do you present the most necessary information at each stage? What is that necessary information?

Through iterating on this problem, they were able to zero in on the solution that you see today. You can observe that the screenshot starts at the largest allowable (and necessary) size, with minimal labeling. The first screenshot presented, has already been reduced slightly in size to accommodate space and address the need for simplicity — presenting just the information necessary to make a voting decision.


Occasionally, to make a voting decision more detail may be needed and is achievable by way of the zoom button.


As the screenshot progresses through the process, clearly less information is needed. The image goes from big to smaller to smaller.


Everything Simpler

Darrell is quick to point out that, while the iterate-to-simplicity process may be most evident in the screenshots, it was universally applied to every aspect of what they did. In this sense, you can see many of the creative influences of CommandShift3 on other sites that they feel epitomize a professional simplicity and/or found ways to present just the most necessary information, such as…

Darrell and the rest of CommandShift3 understood that they were designing for the design community — and this audience always notices every last detail.

Some more of those details…

  • URLs. They needed to be short, simple and bookmark-able.
  • Colors. They understood that the CommandShift3 audience would instantly spot if a color was slightly off or if there was some other mismatch.

“Simplify. Simplify. Simplify.” the philosophy expressed by Darrell Silver in my interview and shared by the other founders (Erin Sparling, Lee Semel, and Amit Gupta) that led them all down a path – of creating a cool product with an awesome (and, in some cases, addictive) User eXperience.

Quick Fact:
Do you know what was the initial inspiration for (click here to find out)

On the Competition…

Another part of CommandShift3’s success was that they were able to identify a completely unserved need within the designer community and come up with a very fun and entirely unique product.


More recently, some competition has been emerging. The most notable competition comes from Technorati founder, David Sifry in the form of You can read more about it here.


Obviously, by just comparing the homepages of CommandShift3 and OKorKO, CommandShift3 remains in a league of its own.

Despite my attempts at getting Darrell to dive into a compare and contrast session or spark a more spirited debate about CommandShift3’s competition, Darrel, flattered, began and concluded his assessment of the competitive landscape when he, with a smile, said, “they have a great idea.”

Time for Making Money!

Nope, and not at all, explains Darrell. CommandShift3 never had any kind aspirations for the level of success that they are currently experiencing — their goal, their singular goal, was, and remains, to serve the under-served design community and provide them with an interactive, online product and community that is both fun and useful. Basically, all they currently appear to care about is not money, but staying focused on the near-term and building a “community that opens up communication channels and lowers the barrier to great design.”

Coming Soon…

Darrel points out that CommandShift3 is all about serving the community. They are a “small team doing something that the Internet allows. That is what gives them the ability to do what they want to do and what other people want them to do.” CommandShift3 won’t “win by locking people in,” but by “being focused and responding to the audience; and being the audience.”

As far as Modular Innovations and data portability, Darrell made it very clear that what mattered first and foremost was SERVING (his emphasis) a community. Evolving within an environment of Modular Innovation and allowing for all sorts of data portability are seen to be very much inline with their community “service.” It was made clear that, as aspects of the site are tweaked and CommandShift3 evolves, people can rest assured that they will be able to own and control the content that they create and contribute to the community at-large. And, as Darrell succinctly put it, “we will absolutely — how could you not be open to letting people keep their data and make it more modular — you are SERVING a community.”

Some of the new features (and tweaks) CommandShift3 will be serving and you can expect to see soon are…

  • Flagging and ranking favorite sites
  • Browsing your “battle” history
  • Looking at what other people like (or don’t like)
  • Observing how popularity of a website changes as the design of that site changes
  • Tweaking some of the language on the site details page to be less confusing. Don’t worry, they are on it.
  • …and a lot more cool stuff.

Printing Screens

CommandShift3, the team that has given new, and social, life to the Print-Screen ‘button’, has truly identified an, until now, unheeded call for a strong, interactive community for all designers and their fans. They have created a product that is fun to experience, interact with, and share through, as well as great for research or exploration, or even testing out new concepts on a large audience.

I had a great time speaking with Darrell Silver and (we all — the fans of CommandShift3) look forward to the many great things coming soon from CommandShift3.

Enjoy & if you haven’t already done so… go and check out! & Have Fun!

Jeremy Horn
The Product Guy

The Top Online News Experience is…

windows_on_monitor_thumb1_thumb1[9](part 5 of 5)….well… keep reading to find out.Over the past few weeks we have all looked at both the Enjoyable and Unsatisfying sides of the top 3 news websites. For those who use them, each individual has found within their favorite source the right mixture of both compelling experience and content that brings them back again and again.Before dubbing the victorious online news experience, I would be remiss to not point out some of the most pressing changes and improvements that all current and prospective news product users would be ecstatic to see…

Yahoo News’ To Do…00_yahoo_homepage

  • Stop concealing the customization options. Allow for easy and obvious customization of the order and inclusion/exclusion of the content on the page.
  • Everywhere, except on the homepage, there is a feature that allows for toggling between headlines only, include summaries, and include photos. However, just like the other customizations, it is very poorly placed — very easy to miss. Yahoo, to help people who are in a hurry, place this feature in an easier to find location and extend this handy feature to the homepage.
  • Eliminating the ‘old style’ feeling website, reducing the aggregator-information overload would go far in improving the overall user experience and enhance the Desirability and Usefulness factors for Yahoo News.

CNN Online’s To Do…00_cnn_homepage

  • CNN has a plenty of good ideas they are working with, but needs to clean up the rough edges of
    • Accessibility is not mentioned enough in reviews of products’ user experience. In this area, CNN does a good job on the article pages, allowing their readers to resize the text with an easy click of the button. This type of functionality should be extended beyond the article pages to all aspects of the site.
    • Page layout and use of whitespace, when used well, can allow for the UX designer to convey a large amount of information within small confines. The page layouts and their usage need to be better utilized to reduce those regions of the page where there is excessive whitespace or labels that unflatteringly wrap to additional lines. Whitespaces (and awkward text layout) distract from the content of the page, often causing the eyes to look at regions of nothingness wondering what article was meant for there.
    • would also benefit from more dynamically loading elements throughout the website. The fewer additional pages that need to load, the faster that the information can be accessed, the better the UX.
    • Better labeling of links that navigate away from to external, non-CNN branded websites and search (that searches the web, as opposed to, by default) would further help reduce unnecessary surprises within the CNN online experience.’s To Do…00_msnbc_homepage

  • In general handles customization of the news experience very well. Customization would benefit from the following improvements…
    • The customization of the news can be found to be scattered all over and requires a great deal of scrolling up and down repeatedly. Allow for a centralized means by which all of the settings can be adjusted
    • Controlling the number of articles to view per section gets too prominent a placement on the page. This is a nifty feature that is easy to use, but surely there exists more useful information or functionality that can be presented in its place (with the control of the number of articles being relocated to the centralized settings control).
  • The Topic links, located in the rightmost region of each page sub-section, present an experience inconsistent with the similar topic selectors within the Explore box at the top of the homepage. When clicking within the Explore box, the article content is dynamically displayed – nice and quickly, no page loads. When clicking one of the Topic links, the user is navigated to a brand new webpage.
    • Enable the dynamic loading of the articles associated with each Topic within the subsection associated with that topic. It will greatly enhance the usability, utility and overall user experience of
  • The dynamic aspects of the news experience can also be further enhanced, through the usage of dynamic mouse-overs for articles. When a mouse lingers over an article for more than a few seconds, a balloon containing some summary information about the article or other related actions can be displayed — empowering the user to take a peak, and deciding the best use of his/her time — deciding which articles call for more/less engagement.


The winner is MSNBC’s online news experience which made excellent use of layout, user control, and dynamic elements – all of which I found maximized the user experience, most especially the Desirability and Usefulness. Of course, everyone has their preferences, their own personal news site winner, and the role of user experience fits into everyone’s decision differently — individually valuing how much weight goes to UX, and how much weight goes to the content or reporters, is unique and personal to everyone.


Does it all really matter? Today, these three news sites remain popular and strong. But, for yourself, or perhaps some or many of your friends, the primary source of news is niche blogs and news sites, frequently accumulated through one or a collection of RSS feeds. Share what parts of your favorite online news experience you enjoy that lead you to prefer one news website over another.

The question, the issue, that the top three, Yahoo, CNN, MSNBC has to solve is how to compete with all of these alternate (and often more expedient) news sources.

How do you see the news sites evolving to compete with the alternate, more flexible, more in control of the users, sources of news?

Will they be able to adapt? (or fade away)

More Information

Enjoy & Discuss & Share!

Jeremy Horn
The Product Guy

MSNBC. An online experience.

windows_on_monitor_thumb1(part 4 of 5) Yahoo News, CNN, MSNBC. Each online news organization presents a different spin on the visual and interactive presentation of the news.Website: MSNBC Online
Briefly: MSNBC’s website was found to be the #3 online news website in December 2007. (read more)

What’s Enjoyable?

  • The primary and category homepages (e.g. Travel, Health, etc.) have simple, clean, and consistent layouts.
    • They provide a minimalist feeling, making for easy skimming of the content.
    • Within the sub-category sections of the pages the link color, mouse-overs, and gradients provide good contrast for eye-easy reading and highlighting.
p01_msnbc_primary p01_msnbc_travel
  • Customization and its impact on the elements on the page(s) is both obvious and easy to use.
  • The category tree allows for the peeling away of the layers of the news-“onion”, gradually displaying more articles within the categories of interest, dynamically and without the need to load (or navigate to) additional web pages.

What’s Disappointing or Unsatisfying?

  • The colors in the page header and footer make those regions of the page very hard to read and, in turn, use.
  • The advertising displayed can be overly intrusive. For example, the ads right in the middle of the text flow of the news articles are too much (and annoying). Don’t you (MSNBC) already have enough ads on the pages?
  • Customizing the desired order of the sections on the page can be a slow and clumsy process. The placement and behavior (one move up or down at a time) are both contributing factors.
c03_msnbc_customize_layout_A c03_msnbc_customize_layout_B

    Until then, think about & share…

    I’ve now explored, in detail, the top 3 online news experiences, Yahoo News, CNN, and MSNBC. Stay tuned to find out which one (or two?) stand out as the best online user experience.

    Subscribe now (click here) to make sure you don’t miss out on the conclusion of this series comparing and contrasting the top 3 online news sites (Yahoo News, CNN, and MSNBC) along with my declaration of online news experience champion.

    Tune in to see if you agree with me — or want to argue a different perspective.

    More Information


    Jeremy Horn
    The Product Guy