It is challenging to concisely and effectively expose students to the social and practical considerations of designing Human-AI systems. But due to curricular or staffing constraints, Human-Computer Interaction (HCI) is often relegated to a single course (or less!) within CS curriculum, leaving little room for some instructors to integrate applied responsible design into existing CS topics. Still, the societal impact of Human-AI interaction deserves both attention and time. To navigate these tensions, I designed a self-contained activity that considers how responsible Human-AI design can fit into existing course structures. The goal of this 1-hour collaborative learning activity is to (1) give students hand-on experience applying ethical design considerations to Human-AI systems, and (2) be highly portable to fit a variety of contexts and time constraints.
The activity: Students use Microsoft’s 18 Human-AI guidelines across four phases (initially, during interaction, when wrong, over time) to evaluate real applications. They complete the following tasks (60 minutes):
• Microsoft’s Human-AI guidelines are presented and distributed to students via printable cards2
• Student groups of 3-5 students are tasked with applying the guidelines to a familiar applications (music recommendations in Spotify, auto-complete in text messaging, Instagram, voice assistants, etc.). Unknown to the students, these applications match those evaluated by experts in Amershi et al.
• Within each group, individual students are responsible for categorizing a subset of guidelines as a violation or clear violation, application or clear application, or not applicable.
• Annotations are made within a collaborative spreadsheet (see https: //bit.ly/hai-activity) to facilitate virtual classroom environments.
• After an individual work period, students come back to their groups to explain, debate, and settle on a full analysis of the group’s application.
•(Optional) Pairs of groups that independently evaluated the same application meet to settle on a final categorization of each guidelines(violation or clear violation, application or clear application, or not applicable)
•Coming together as a class, expert analyses from Amershi et al. are revealed and compared with student evaluations.
•The instructor guides group discussions about why expert decisions may differ from student evaluations, and whether the guidelines adequately capture the full scope of student experience with the applications.
While relatively simple, the applied nature of this activity yields rich discussions about ambiguities and challenges surrounding the design ofHuman-AI systems in a relatively short amount of time (a single class period).Extensions of this assignment can be used applied to emphasize corporate responsibility and industry design incentives
-
introduction: I found that giving a light introduction to the guide- lines prompted more significant peer-learning opportunities be- tween students. Depending on the background of students, instruc- tors might want to consider more significant initial scaffolding.
-
class size: This activity has primarily been used in classes that contain between 25-30 students. However, due to the collaborative structure, it should be able to scale to larger (or smaller classes) by changing the number of applications or the number of groups who are devoted to each application. For example, Amershi et al. contains 10 different applications [1]. If two student groups (of 4) are assigned to each application, the structure would still hold for a class of 80.
-
background: This activity has been run in a Human-Computer Interaction course that targets 3rd and 4th year undergraduate Com- puter Science students. This background enabled me to move quickly through initial introduction of the guidelines. Students with less background may need slower introduction or a rephrasing of the terminology.
-
individual component: I would encourage instructors not to skip the assignment of roles within groups. I found that having each student within a group be specifically responsible for a subset of guidelines yielded more student participation in both group and class discussions.
-
context: This activity was originally designed for a remote class- room experience (necessitated by the COVID-19 pandemic), and groups were facilitated using breakout rooms in Zoom. A physical classroom may yield other opportunities for collaboration on shared tables/whiteboards beyond the spreadsheet shared here.
This activity touches on a number of evidence-based learning approaches within a relatively short amount of time.
- collaborative learning: individual students use peer-learning to apply guidelines to an application area and then share those guide- lines with larger groups. This is reinforced when student groups meet with each other to resolve differences in evaluations
- using meaningful and relevant context: students consider and critique AI within the context of applications that they have mean- ingful interactions with on an everyday basis.
- culturally relevant pedagogy: guidelines such as Mitigate Social Bias or Match Relevant Social Norms necessitate discussions sur- rounding issues of diversity and social responsibility, and can guide conversation about the sociopolitical role of AI and design.
Potential Modifications: There are a number of modifications that can be made depending on the structure of the course or the goals of the activity.
-
Corporate comparisons: Google and Apple have also created publicly- available guidelines to navigate machine learning or AI interactions with people [2]. One modification of this assignment would be to as- sign different guidelines to different groups for the same application (for example, music recommendation is evaluated by Microsoft’s guidelines in one group and Google’s guidelines in another group). Subsequent group discussions can focus on why evaluations differ between industry guidelines and how those guidelines may nudge design.