User Testing Steps

Mobile Application Usability Testing Process

Step 1: Preparation

Using your interactive prototype(visual mockups) with your test plan, conduct a usability test with 3 test participants for the 2 scenarios: your own scenario + one additional scenario

• Resource: Testing Documentation templates (opens new window)

1) Scenario One: In your Usage Scenario document

• Summary: In your Usage Scenario document
• Tasks:
  • browses _______
  • navigates to ________
  • do something specific
  • navigates to ________
  • do something specific

2) Scenario Two: In your Test Plan

• Summary: In your Test Plan
• Tasks: In your Test Plan

Step 2: Testing

1. Introduction:
   • introduce yourself
   • promise confidentiality to the user (Sign NDA)
   • explain what you will be testing and the test method
     • You are testing the product, not the users.
     • Have users trying some features to help you identify usability issues so that you can improve the app user interface to achieve better user experience.
     • Think aloud method: Ask users to say what they are looking at, what they are trying to do, and what they are thinking.
2. Collect Demongraphic information:
   • Take the participant's demographic information: Full Name, Gender, Age Range, Occupation, Browsing Device, Technical Ability etc.
3. Assign the tasks (scenarios from your Test Plan/Test Script)
   • The test user walks through test scenarios.
   • The researcher observes the performed task and takes notes on the steps taken by the user.

Note: Whenever possible, record each test session.

Generic questions to ask:

Before Task (profile questions):

• What do you do for living? What are your hobbies?
• What kinds of internet searching do you do on your phone vs your computer?
• Can you think of the last time you looked for a cleaning service?

Good opening questions (by Steve Krug): Please take a look at this app and tell us what you think it is: What’s the first thing you notice? What can you do with this app? What products or services are offered? Who is this app intended for? Just look around and speak everything that comes to your mind.

During Test:

• What are you thinking right now?
• Can you tell me more about what you just did?
• I notice you [did something] - can you explain that to me?

When struggling:

• Where would you tap (click) in this case?
• How do you think that would work?
• How might you get assistance to figure this out?

After Test:

• I noticed you had an issue with [something], what would have made it easier for you to understand?
• Do you have any questions for me?

Step 3: Document

• Take detailed notes of each user's behaviour, comments, errors and completion (success or failure) Note: Use Quantitative Metrics from your Test Plan
• Review the errors or issues found to identify any required changes to the Visual Design and/or your Interactive Prototype(visual mockups)
• Summarize each test
  • ParticipantID (e.g. Tester 1)
  • Timecode of the issue (When have a video recording of your session, E.g. 24:12)
  • A brief description of the issue
  • The area of the site where the issue occurs (e.g. product detail page)
  • Possible solutions
  • Severity rating

Step 4: Report

• Summarize your test findings and analysis report and add them to your final project slide.
• Revise your UI design and prototype(interactive visual mockups) accordingly for the final project submission.
• Usability Test Results & Analysis slide content will include these 3 Questions and Answers
  1. What issues did the participants have while interacting with the prototype(visual mockups)?
  2. Based on the issues, what changes did you have to make to your visual design & interactive prototype(visual mockups) to improve usability?
  3. How does your App meet the persona goals?

LOGISTICS OF UT

Before Tests

• Ensure your prototype supports your test plan & script
  • Include sufficient screens/states in the interactive prototype (visual mockups)
  • Ensure your script acknowledges gaps in functionality (404 coming soon)
• Test material: Test plan, prototype, and test script & test notes
• DO A PILOT TEST!! (aim for approx 20 mins test e.g. with your classmate/test partner)

Test

• Decide if you want to take your own notes, or partner up with a classmate to take notes for each other.
• Find a participant.
• Run your test, while your note taker takes notes.
• Try to limit it to about 20 mins.
• When finished, take 5 mins to sync with your note taker (if you have one) or just jot down any additional notes
• Repeat with the next participant! (Take turns if you have a partner)
• Make sure you each perform your test on 3 participants

Best Practices

• Make sure your participant feels comfortable.
• Silent pauses are ok. Focus on watching and listening.
• Remain neutral. Do not show disappointment if tasks fail.
• Avoid leading questions or comments.
• If participant asks for help, start with “What do you think?”
• Stick to the script!
• Watch the clock!