User Experience & Interaction Design for AR/VR/MR/XR

This module will provide you with an introduction to XR design and development. When I say design, I usually mean user experience design or interaction design. Some designers have quite specific definitions of each of those terms, but I think of them broadly as forms of design focused on the creation of new user interfaces. While I think of many parts of this course as providing the theory and background to XR design, I will focus on the practical aspects of XR creation. I will start by making an analogy to filmmaking to establish key requirements, before going on to talk about the main steps part of a larger process and how to get started with doing XR. As we are just getting started, this first module does not have an honors track component associated with it. The honors track will start in the second module.

The second module will introduce you to design thinking and how common methods and techniques can be adapted for XR. This will help us build the right mindset for designing novel and compelling XR experiences. We will talk about building empathy with your users, ethical and responsible design, and best practices and design guidelines. I will also cover how to conduct and formulate design critiques in an effort to learn from existing interfaces. Finally, we will look at design jams as a way of solving design challenges collaboratively. Our first exercise in the honors track associated with this course will be a design critique of a VR or AR expedition as part of Google’s Expeditions XR app. You will be asked to develop one new idea that you will then take forward as you start prototyping your design concept next week.

We are moving from the conceptual to the practical, from thinking to doing. The next two parts of this course are dedicated to prototyping. In this first part on physical prototyping, I will emphasize paper a lot and also talk a lot about sketching. In my mind, brainstorming translates to sketching alternate ideas. Rather than creating, for example, a mind map, I sketch to ideate. This is a practice that I will emphasize. We’ll start flat on paper but will quickly become more physical using specific paper templates and dioramas. We’ll also see how to create mostly physical prototypes, but how to enhance them with AR techniques, to move closer to our envisioned final XR experience. In the honors track, you will have two major exercises. You will first create an XR storyboard based on the XR app you have previously critiqued. The design critique and storyboarding will help you develop a new idea of improving the existing XR interface. You will then prototype your design concept using physical materials including paper, cardboard, Play-Doh, and transparency. I hope you will appreciate the freedom and flexibility that physical prototyping gives you, allowing you to create a new XR experience without many of the constraints of digital tools.

The last module of this course will cover digital prototyping and user experience evaluation techniques for XR. While it is recommended to start prototyping on paper, it is crucial to experience a design concept for XR with XR technologies. I will introduce you to digital prototyping techniques as a way of increasing the fidelity of your prototype designs. You will learn about digital prototyping tools that support creating initial VR or AR experiences without the need for programming. Some tools support immersive authoring, which enables prototyping of XR experiences directly inside VR or AR. We will also learn about usability evaluation techniques and how to design and conduct a user study. Our final lecture will be on creating your XR portfolio as a way of effectively communicating and showcasing your design work. In the honors track, we will finalize our XR project with a digital prototype and submit our work for peer review. The peer review will not only provide important feedback on your work but also allow you to review and critique your peers’ work in a constructive way. I consider formulating good feedback and acting on critical feedback two vital skills for any designer but especially for those working in a space as rapidly evolving as XR.