7 usability testing mistakes

From an article on the "user interface engineering" website:
When a design has a usability problem, it's because someone made a wrong decision. They chose to take the design in a direction that creates frustration for the user. A different design choice would have prevented the frustration.

We consider a usability test to be successful when the design team members receive the information they need to make the right decision. Successful usability tests produce informed decisions.

There are two outcomes from poor decisions: either the user experience is worsened because of a change that just shouldn't have happened; or a valuable opportunity is missed to improve the design's user experience. Either way, when usability tests work, these results are significantly less likely.
  1. Not knowing why you're testing
    1. It's not about how the user "feels"
    2. Its about telling where the UI causes frustration.
    3. Pose behavioral questions
    4. "Usability testing is all about seeing the design through the eyes of
      the test participants. As they work their way through the design, you
      get to see and hear what works well and where it becomes frustrating to
      accomplish their goals."
  2. Not bringing the team together
    1. Do the test nearby the team.
    2. Video the test
  3. Not recruiting the right participants
    1. Don't focus on demographic (age, income)
    2. Do foxus on distinctions that make users behave differently (fluency in the content area).
    3. If they don't have the right experience, they'll get stuck in places where the real users will breeze past.
    4. "What attributes will cause one user to behave differently than another?"
  4. Not designing the right tasks
    1. Users want to please you by following directions, so make it a bit more freeform.
    2. "You can get around this mistake by constantly exploring the "context of
      use." When designing tasks, ask yourself, "What events or conditions in
      the world would motivate someone to use this design?"
  5. Not facilitating the test correctly.
    1. Not boring for participants or team members
  6. Not planning on sharing the results
    1. Get the info to the design team
    2. Reports don't work very well - they don't get read
    3. Instead use "review sessions that happen right after each test, starting an email
      discussion list to talk about the test and various interpretations, and
      interactive workshops to review the design and what we learned"
  7. Not iterating to test potential solutions
    1. "Usability Testing is great for identifying problems. Yet, it's horrible at identifying solutions."
    2. "we've never run into a design team that couldn't generate a half dozen
      possible solutions to any problem, within moments of its discovery."
    3. "Plan a round of testing, to validate any yet-to-be-discovered potential solutions."

1 comment:

Alexandra said...

I think you make some very valid points. Websites are designed by people who use the internet a lot, so what seems simple and intuitive to someone who is tech savvy may be confusing and counter-intuitive for end users who spend most of their time online looking at Lolcats. If a large part of your target audience will be people who think the internet is for porn and chat, you would want to make sure that this type of end user is comfortable with the UI.

Also having too much direction is "leading" the user. When the typical end user visits your site they won't have a list of actions to take, they will just have to figure out what to do from site navigation. Providing too much direction can skew the results in favor of a preconceived idea, instead of actual user feedback.