Course Level
Other
Collection Item Type
Other Material Type
Synopsis

Usability testing is a key research method in human-computer
interaction (HCI). When students are designing for others, usability
testing is an opportunity to learn how the design is currently
working and how it can be improved. This usability testing plan
template gives individual students or teams a structure to help plan,
conduct, and analyze data from a study. The template walks
students through the process of planning a study through a series of
questions and planning materials. The template is especially helpful
for students new to usability testing and can be adapted and
adjusted as needed.

ACM Digital Library Entry

Recommendations

The following section includes recommendations for incorporating
the usability testing plan into an HCI course.

Implementation
You can incorporate the usability testing plan into most human-
computer interaction courses. Usability testing is one of the most
common user research methods. Providing an opportunity for
students to learn this method will help them develop their skills as
technology designers, developers, and researchers.
While the usability testing plan is flexible and adaptable for
many contexts, I see two primary uses of it in an HCI course. The
first is conducting a usability test on an existing product. The main
goal of testing an existing product is to better understand what
usability testing is as a method, when it is an appropriate choice of
method, and how to conduct a usability test. By testing existing
products, students can start to gain insight into the difference
between their assumptions and learning about other peoples’
experiences. In this case, the instructor could focus on what
usability testing is, what types of data it helps produce, and how
testing with representative users can help reveal usability problems.
It can be helpful to choose one product that the entire class can investigate or similar products so a class can identify specific
challenges across the same domain. While a usability test can be
conducted on a variety of technology products, such as software,
websites, or mobile apps, choosing one to focus on as a class can
be helpful. For example, if the goal is to investigate complex
information-rich websites, it can be helpful to choose either one or
an array of government websites (either local, regional, or
national). Students can then conduct usability tests and discuss
similarities and differences.
Another approach is to use usability testing to investigate how
products may be excluding certain audiences. This can be done by
conducting an accessibility audit as part of the usability testing to
examine how a product may or may not support users with
disabilities [8,12] and discuss the connections between UX and
accessibility. Further, examining a product to ask who is left out or
harmed by the design can be a fruitful area of inquiry. For example,
Bartolotta investigates the usability of a document designed by the
US Department of Homeland Security communicating about the
“Family Separation Policy” which explores the limits of usability
testing when examining oppressive artifacts [2].
The second option is to conduct a usability test as part of a
class where students are creating or designing new systems,
interfaces, or technologies. In this case, the usability testing plan
can be used during an evaluation phase of the project. Students can
plan the study to test their own design, gather data, and use that data
to make changes and iterations to the original design. In this second
situation, learning how to conduct a usability test and then also how
to make improvements to a specific design is the focus. In this case,
students learn about the method of usability testing and can also use
the results of the study to make concrete and meaningful changes
to their designs based on the results.
In each case, instructors should plan between 2-5 weeks for a
module on usability testing. Students will need to be introduced to
what usability testing is and then given time to design and plan their
study, recruit users, run sessions, and analyze the data.
 

Working in teams
I highly recommend that students complete the usability testing
plan as a team, both because it is a high-impact educational practice
[7] and because it helps to highlight the complex communication
ecology present in industry and research contexts. Many of the
items in the plan are questions that need discussion and negotiation,
namely, who are the users, which users are the focus of the study,
and what usability concerns or issues should be investigated as part
of the study. In addition, planning and running a study in a short
amount of time is a considerable amount of work, and having clear
roles and responsibilities on the team can help share the workload.
For example, when planning the study, students can divide the
workload of creating the materials they need for the study,
including a consent form, script, scenarios, and questionaries.
When conducting the study, students can choose designated roles:
recruiter, facilitator, note-taker, technical specialist. Students can
take on more than one role. Alternately, if an important learning
goal for students is to experience each role, you can have them take
different roles for each participant in the study. For example, if one
student is the facilitator for one session, they can be the notetaker
for the next session. While this approach might sacrifice
consistency or rigor for the study, it provides the students with a
broader learning experience.
 

Foregrounding ethics, diversity, inclusion
Usability tests can be a rich site for engaging students in
conversations about ethics, diversity, and inclusion. When introducing usability testing, make sure to spend time discussing
research ethics including how to interact with participants and
issues of consent and confidentiality. Start by introducing students
to the Belmont report [13]. Review the materials provided by your
schools’ Institutional Review Board about conducting research
with people. In addition to research ethics, it is helpful to introduce
students to ethics specific to HCI and UX. Consider introducing
students to UXPA’s Code of Professional Conduct [14], Nielsen
Norman Group’s Ethical Maturity in User Research [9], and
ACM’s Code of Ethics and Professional Conduct [15]. Introduce
and encourage students to use consent forms for their usability
studies. A sample one is included with the Usability Test Plan.
Additional readings and resources can help disrupt the idea of
an average user and highlight the diversity of people who will be
using the products that professionals design and therefore should
be included in user research such as usability testing. Examples
include, Isaacson’s work on Inclusive Research [6], Holmes on
how inclusion shapes design [5], and Wachter-Boettcher on the
danger of defaults [11]. Reviewing and discussing these works as
part of an HCI course that features usability testing would
strengthen students understanding of designing for inclusion.

Recruitment
In addition to thinking about diversity and inclusion, students
should recruit a diverse group of representative users for their
usability studies. Recruiting can take significant time and
coordination, so students should start early. It is tempting to recruit
students in the same class or same program, but the experience will
be more authentic and robust if the participants recruited are
representative users of the product and are not familiar with the
practice of usability testing. If students are working with a project
partner, like a community group, campus partner, or client, ask
them for help recruiting participants. If not, encourage students to
consider creative ways to find people who are representative users
and strive to recruit a diverse group of users. Ideas include locating
affinity groups through online forums, such as social media or
meetups, or community organizations like a library or a community
center.

Assessment
Planning and conducting a usability study is an opportunity for
students to learn about both design and research. There are several
opportunities to assess student work. I recommend a formative
approach to coach students in a way that helps them reflect on their
choices while also attending to the logistical planning needed for a
study. You can ask students to complete specific sections of the
plan and provide iterative feedback or ask students to submit the
completed plan. In both cases, I recommend providing clear
deadlines or gates so students cannot start their recruiting or testing
until they have received instructor feedback.

Adaptions and additions
The usability testing plan is a flexible tool that can be adapted and
supplemented with additional materials to fit specific course
context. Readings and additional templates can be linked within the
document to provide students with more structure to their study.
For example, consider using industry standards scales in a post-
study questionnaire such as the System Usability Scale [3]. Further,
depending on the instructional goals of the course, instructors could
spend additional time on concepts such as research ethics, study
facilitation, data analysis, or reporting.

 

Engagement Highlights

The usability testing plan template helps enact several engagement
practices. First, it helps to build student confidence and
professional identity by sharing best practices in planning studies
with students that are used in industry. Students can complete the
usability testing plan and submit for review from an instructor who
can give effective encouragement to students by praising effort and
offering constructive feedback. Second, the template, enables
assignments that use meaningful and relevant content, students can
perform usability tests on their own emerging designs or choose an
existing system that they are interested in learning more about. In
this way, the template can enable student agency and choice by
encouraging students to select their own topics of inquiry while
creating a standard format for students or student teams to compare
their results and outcomes from their usability tests. Third, since
usability testing is inherently focused on the users of a design or
system, the template encourages students to consider a broad range
of people who might be using the product and highlights their
needs.
 

Computer Science Details

Programming Language
None

Material Format and Licensing Information

Creative Commons License
CC BY-SA