(Almost) Live from Salisbury’s Web Communications and Strategies Conference: how to do user testing with $5 and 5 people

July 20th, 2005 Karine Joly 5 Comments

Even if it’s somehow easy to forget, we do design websites for users.

So, why user testing isn’t a bigger part of our job?

I know the answer and so do you: time and money, which are definitely limited resources in our field.
We don’t have the kind of budget necessary to pay usability experts, nor do we have the time or staff to plan and implement a good website usability study. Or, so we think.

In her presentation “Got Five? Effective User Testing with $5 and Five People,” the director of Web Communications at Texas Woman’s University, Liz Norell, showed that we’re all very good at finding excuses to avoid learning the truth about our websites and how wrong our design assumptions can often be — obviously, I’m not talking here about those of you who have been doing user testing for ages or years.

In a lively and well-documented session at Salisbury’s Web Communications and Strategies Conference, Ms. Norell didn’t hesitate to share all her secrets with her peers, as well as the title of the book that made her life much, much easier: the excellent “Don’t Make Me Think” by Steve Krug (I read the book a couple of months ago, and this is definitely one you need to have if you’re about to start a redesign project).

Before presenting her 11-step plan to a better site, Liz Nowell explained that the goal of user testing is simply to answer the following questions:

  • Can people find what they’re looking for on the college/university website?
  • How quickly can they navigate to complete a specific task?
  • Have the web developers and designers correctly anticipated user behavior?

  1. Identify the areas you need to test
    • Define the scope (university web, department, page)
    • Use a committee to identify areas and scope
    • Give your committee the opportunity to give feedback even if you don’t plan to use it.

    Obviously, you’ll want to test the most mission-critical parts of your website such as:

    • Admissions
    • Financial aid
    • Degree/Programs
    • Registrar/registration (last day to drop, payment deadline, how to register)
    • Student services (library, dining services, residence hall)

  2. Draft questions
    • Write specific open-ended questions that require students to look at key content areas
    • Find answerable questions, but you can include some difficult or tricky one to avoid educated guess
    • Circulate questions among stakeholders for comments about wording not content
    • Test the questions on a colleague or friend to make sure the question makes sense

    For example: What is the financial deadline for applying for fall 2005 financial aid?

  3. Schedule testing

  4. It’s important to test any changes you made, so you will need to test often. Five tests should take about an hour for 5 tasks per student who does the test.

    • Schedule 3 iterations of testing that are approximately 1-2 weeks apart as you will need enough time between iterations to make adjustments
    • Pick a time when students aren’t rushed (orientation – after registration), avoid finals
    • Test on a Monday or Tuesday (last ones are very good – before lunch)

  5. Get your team ready

  6. It can be done by just one person, but will be easier with two people. Use your student assistant (if you’re lucky enough to have one ;-) to recruit other students.

  7. Pick the right location

  8. For obvious reasons, a computer lab is probably the best spot. You should try to offer some privacy for students who want it, but let others talk together if they prefer.

  9. Recruit Lab Rats students
    • Don’t forget signage, it’s critical.
    • Offer free chilled water or free pens
    • Approach people who don’t appear rushed
    • Be clear you need their help
    • Don’t recruit more than 2 students for each observer

  10. Conduct the test
    • be quiet, don’t say anything
    • Remind the testers that nothing they do is wrong and if they can’t find the answer, it’s your fault not theirs. Ask them to move on to the next question, if it takes more than three minutes to find the answer (this is your SAT not theirs)
    • Give them a document with task-oriented questions and optional questions about their general likes/dislikes (optional)
    • Keep track of time – what time they start and stop
    • Keep track of their clicks by writing notes / use screen capture software to record their paths

  11. Complete the test
    • Remind testers about the optional questions
    • Thank them and don’t forget to give the freebees and your business card so they can contact you if they have questions
    • Tell them that you appreciate the time they spent to help you, but keep it casual

  12. Transcribe notes
    • Rewrite quickly your notes into discernible navigation paths
    • Analyze the data and see if you can spot any trends

  13. Write a short report
    • Summarize the main findings
    • Identify the problems you observed
    • Prioritize and editorialize on applicable issues
    • Share the report with your committee and don’t forget to keep a copy

  14. Make changes
    • Pick your battles: choose the issues that can be worked out
    • Set a timetable
    • Repeat

And, here you go a great and simple way to find out what users can really do on your website.

Want more details and see Liz Norell’s questionnaire and report? Just go to this web page dedicated to user testing and set up for the conference’s attendees

5 Responses

  1. […] I’m just back from Salisbury where we had a great conference. In my previous post, I talked about what was probably the best presentation I attended there: “Got Five? Effective User Testing with $5 and Five People” by Liz Norell, the director of Web Communication at TWU.

    As she indicated in her presentation, her method has been largely inspired by Steve Krug’s “Don’t Make me Think.” […]

  2. […] Liz Norell gave one of the most interesting presentations I attended at the Web Communications and Strategies Conference in Salisbury this year […]

  3. quan vu says:

    This year was my first experience at SU web conference and I really thought it was great and this blog is a great way to share information. Thanks so much. Quan Vu

  4. Karine says:

    Thanks for your nice words about the blog, Quan! Feel free to comment when you stop by.

  5. […] users will provide you great insights to improve the usability of your website (Gosh, I even wrote a post about a conference session about this — 5 years ago ;-). That’s why there is no reason for skipping Do-It-Yourself usability […]

Got a question or comment?