Beginning Windows Mixed Reality Programming: For HoloLens and Mixed Reality Headsets

Description

Develop applications and experiences for Microsoft’s HoloLens 2 and other Windows mixed reality devices. This easy-to-follow guide removes the mystery behind creating amazing augmented reality and virtual reality experiences. Mixed reality development tools and resources are provided. Beginning Windows Mixed Reality Programming, 2nd edition clearly explains all the nuances of mixed reality software development. You will learn how to create 3D objects and holograms, interact with holograms using voice commands and hand gestures, use spatial mapping and 3D spatial sound, build with toolkits such as Microsoft's Mixed Reality Toolkit and Unity’s AR Foundation and XR Platform, create intuitive user interfaces, and make truly awe-inspiring mixed reality experiences. This newly revised edition also includes updated content for HoloLens 2 development, including tutorials for new interactions such as hand tracking and eye tracking. What You Will LearnPrototype ideas quickly for the HoloLens 2 and Windows mixed reality devicesGet started with Unity, the preferred tool for developing 3D experiencesLocate and import 3D models for your project, or make your ownUse spatial sound, voice commands, gestures, hand tracking, and eye trackingBuild with Microsoft's Mixed Reality Toolkit and other toolkits to make apps the easy wayPublish to the Windows Store and make money from your app Who This Book Is For Programmers looking to quickly learn how to create experiences for HoloLens 2. Also for programmers interested in building applications for the growing crop of virtual reality devices that support the Windows Mixed Reality platform.  Table of Contents About the Authors About the Technical Reviewer Acknowledgments Introduction: The Mixed Reality Future Part I: Getting Started Chapter 1: Gear Up: The Necessary Hardware and Software Tools Making Sure Your PC Is Ready Understanding the HoloLens and Other Windows Mixed Reality Hardware Inside-Out Tracking and Spatial Mapping Spatial Sound Transparent vs. Immersive Headsets Downloading and Installing the Required and Optional Software Tools Installing Visual Studio Installing Unity Downloading the Mixed Reality Toolkit Summary Chapter 2: Unity Crash Course What Is Unity? Free vs. Paid Tiers of Unity Your First Unity App Step 1: Create a New Unity Project Step 2: Save Your Scene Step 3: Create a Ground Plane Step 4: Rename Your Plane Step 5: Reset Ground Plane Position Step 6: Zoom to Your Ground Plane Step 7: Scale Your Ground Plane Step 8: Create the Ball Step 9: Rename Your Ball Step 10: Reset the Ball’s Position Step 11: Zoom to Your Ball Step 12: Raise the Ball’s Position Step 13: Color the Ground Blue Step 14: Add Physics to the Ball Step 15: Enable Keyboard Control Step 16: Testing Your App Summary Part II: Building Holographic Experiences Chapter 3: Creating Your First Hologram Getting Unity Ready for Mixed Reality Development Step 1: Import Mixed Reality Toolkit to a New Unity Project Step 2: Use Mixed Reality Toolkit to Prepare Your Scene for Mixed Reality Development Your First Hologram Step 1: Create a Cube Step 2: Zoom to Your Cube Step 3: Move the Cube Away from the Camera Step 4: Resize the Cube Step 5. Test Your App Step 6: Install Your App on the HoloLens Summary Chapter 4: Introduction to the Mixed Reality Toolkit What Is the Mixed Reality Toolkit? The Three MRTK Repositories Mixed Reality Toolkit Setup 1. Importing MRTK Asset Files 2. Unity Package Manager Mixed Reality Toolkit Components MRTK: Input System How to Run a Test Scene MRTK: Hand Tracking MRTK: Solvers MRTK: Multi-scene Manager MRTK: Spatial Awareness MRTK: Diagnostic Tool MRTK: Boundary System MRTK: UX Controls MRTK: Camera System MRTK: Profiles MRTK: Standard Shader MRTK Online What Is GitHub? MRTK Help and Documentation Summary Chapter 5: Interacting with Holograms Input Methods Gaze Tutorial Step 1: Set Up Unity Scene Step 2: Add Scenes to the Build Menu Step 3: Try the Scene! Step 4: Understanding the Scene Step 5: Use Gaze in Your Project Gestures Tutorial Step 1. Load Test Scene Step 2. Try It Out! Step 3. Bounds Control Step 4: Press and Touch Interactions Step 5: Object Manipulator Step 6: Implementing Gestures in Your Application Voice Command Tutorial Step 1: Load the Example Scene Step 2. Try It Out! Step 3: Understanding the Scene Step 4: Add Your Own Voice Command Step 5: Using Voice Commands in Your Own Project Best Practices for Voice Commands Other Hardware Input Summary Chapter 6: Using Spatial Awareness What Is Spatial Awareness? Spatial Awareness Tutorial Step 1: Set Up Unity Scene Step 2. Try It Out! Step 3. Understanding the Scene Step 4. Using Spatial Mapping in Your Application Occlusion Step 1: Apply Occlusion Step 2: Try It Out! Scene Understanding Summary Chapter 7: Spatial Sound Spatial Sound Tutorial Step 1: Set Up Unity Scene Step 2: Try It Out! Step 3: Understanding the Scene Step 4: Enabling Spatial Sound in Your Application Spatial Sound Design Considerations When to Use Spatial Sound What to Avoid When Using Spatial Sound Summary Part III: Growing As a Holographic Developer Chapter 8: Azure Spatial Anchors What Is Azure Spatial Anchors? Azure Spatial Anchors Tutorial Step 1: Creating new Unity Scene Step 2: Installing Inbuilt Unity Packages Step 3: Download and Import Tutorial Assets Step 4: Preparing the Scene Step 5: Configuring the Buttons to Operate the Scene Step 6: Connecting the Scene to the Azure Resource Step 7: Test the Application in Device Summary Chapter 9: Shared Experiences What Are Shared Experiences? Multiuser Capabilities Tutorial Setting Up Photon Unity Networking Step 1: Creating New Unity Scene Step 2: Enabling Additional Capabilities Step 3: Installing Inbuilt Unity Packages Step 4: Importing the Tutorial Assets Step 5: Importing the PUN Assets Step 6: Creating the PUN Application Step 7: Connecting the Unity Project to the PUN Application Connecting Multiple Users Step 1: Preparing the Scene Step 2: Create and Configure the User Step 3: Create the Avatar Step 4: Create the Prefab Step 5: Configuring PUN to Instantiate the User Prefab Sharing Object Movements with Multiple Users Step 1: Preparing the Scene Step 2: Configuring PUN to Instantiate the Objects Integrating Azure Spatial Anchors into a Shared Experience Step 1: Preparing the Scene Step 2: Configuring the Buttons to Operate the Scene Step 3: Connecting the Scene to the Azure Resource Trying the Experience with Spatial Alignment Summary Chapter 10: Awe-Inspiring Experiences What Makes an App Awe Inspiring? Optimization and Performance How to Monitor for Performance Best Practices for Performance Start Monitoring and Optimizing Early Optimize Polygons and Textures Level of Detail Rendering Culling Enable Single-Pass Instanced Rendering Optimize Shaders Simplygon Stabilization Plane Design and Magic Best Practices for Design Spatial Mapping Distance from User Shadows Voice Sharp Text Bounds Controls Toolbars Colors Design Experiences to Avoid Additional Resources Adding Magic: Vuforia Step 1: Install the Vuforia Sample Step 2: Try It Out! Summary Chapter 11: Turning Holograms into Money Publishing Your App to the Microsoft Store Freelancing and Contracts Where to Find Mixed Reality Freelance Opportunities Increasing Your Chances of Winning a Contract Future Opportunities Today Summary Chapter 12: Community Resources HoloDevelopers Slack Team What Is Slack? What Is the HoloDevelopers Slack Team How to Join the HoloDevelopers Slack Team Participating in the HoloDevelopers Slack Team Other Online Communities and Resources HoloLens Developers Facebook Group Unity and Unity HoloLens Forum HoloLens Subreddit Next Reality News YouTube Local Events and Meetups Europe Meetups North America Meetups Asia Pacific Meetups Hackathons Notable Industry Events Summary Index