Developing AR/VR/MR/XR Apps with WebXR, Unity & Unreal

This first module will cover three main approaches to XR development: WebXR, Unity, and Unreal. Each approach requires different development tools, programming languages, and technical skills. They also differ in the fidelity and quality that can be achieved and in how many devices and users can be reached with each. In addition to getting a fundamental overview of XR development platforms and tools, a main goal this week is to make the jump from 2D to 3D: many of our learners will be familiar with web and mobile development, but 3D development is quite different. In the honors track, I will guide you through the creation of your first 3D scene using WebXR, Unity, or Unreal. I will also provide first steps instructions in each of them, allowing you to analyze and compare before choosing your approach and tools.

This second module will focus on the development of VR applications. We will learn what it takes to create a virtual reality, about menus and navigation techniques in VR, as well as object selection and manipulation. I’ll share various examples that I have implemented using WebXR to illustrate the concepts. I will also cover approaches in Unity and Unreal. You will also see me give demos directly in VR to better understand how navigation and interactions work in VR. Many of the lessons are rooted in a VR design case study inspired by the Detroit Zoo, which my student Kara Dailey and I created specifically for this XR MOOC. In the honors track, I will guide you through the creation of your first VR scene using A-Frame, Unity, or Unreal. And, I will provide more hands-on lectures on how to create basic and immersive VR scenes with WebXR using A-Frame.

This third module will focus on the development of AR applications. We will learn about marker-based and marker-less approaches to augmenting the environment, about the design and role of markers for prototyping and deployment, and about the differences to marker-less AR. Again, I’ll share various examples that I have implemented using WebXR to illustrate the concepts. I will also cover approaches in Unity and Unreal. We will also learn about the differences in design between hand-held and head-worn AR. Many of the lessons are rooted in an AR design case study bringing Kepler’s Laws of Planetary Motion, which my student Shwetha Rajaram and I created in an independent study project. In the honors track, I will guide you through the creation of your first AR scene using A-Frame, Unity, or Unreal. And, I will provide more hands-on lectures on how to create marker-based and marker-less AR scenes with WebXR using A-Frame.

This fourth and final module will broaden the scope of this course by introducing you to advanced techniques and also research in XR. In terms of advanced techniques, we will learn about procedural generation, redirected walking, and custom controllers for VR. I will also cover 3D reconstruction, object recognition, and custom displays for AR. I will include examples from recent research. We will also go over important topics such as accessibility, collaboration, personalization of XR experiences. These topics are receiving increased attention in current research and will significantly shape the XR landscape in the next five to ten years. I will also give a lecture on XR research, where I’ll use one of our recent research projects and present the many iterations through which it went based on user and system performance evaluations. I will also talk about good and bad research questions, which mostly has to do with scoping and how far you want to push the envelope. Finally, I will share my Making of the XR MOOC through which I hope to share some of the techniques I’ve employed in giving, recording, and sharing demos with AR/VR technologies. In the honors track, we will conclude our projects with a peer review of 3D, VR, or AR scenes.