Real-Time Video Content for Virtual Production & Live Entertainment: A Learning Roadmap for an Evolving Practice

Description

Real-Time Video Content for Virtual Production & Live Entertainment looks at the evolution of current software and hardware, how these tools are used, and how to plan for productions dependent on real-time content. From rock concerts to theatre, live television broadcast to film production, art installations to immersive experiences, the book outlines the various applications of real-time video content – the intersection of gaming and performance that is revolutionizing how films are made and how video content is created for screens. Rather than render out a fixed video file, new tools allow for interactive video content that responds to audience activity, camera position, and performer action in real time. Combining software renderers with environmental information, video content is generated nearly instantaneously to simulate depth, creating a new world of Virtual Production. This book provides an overview of the current software and hardware used to create real-time content while also reviewing the various external technologies the real-time content is dependent upon. Case studies from industry experts appear in each chapter to reinforce the tools described, establish industry practice, and provide insight on a complex and rapidly growing discipline. Real-Time Video Content for Virtual Production & Live Entertainment prepares students and practitioners for a future working with real-time technologies and informs current entertainment technology professionals how to rethink about their old roles using these new tools. The book includes access to a companion website featuring web-based and video resources that expand on topics covered in the text. Each chapter has a unique page that points to example material, video presentations, and professional studies on chapter topics. You can visit the companion website at rtv-book.com. Cover Half Title Title Copyright Contents List of Contributors Acknowledgments Chapter 1 Introduction We Are Surrounded by Visual Content Creative Video for Live Visuals Live Entertainment Is Forever Changed Real-Time Video Content Is the Engine that Powers Virtual Production What This Book Is Not Section 1 Exploring Real-Time Content Chapter 2 Defining Real-Time Video Content Defining the Term “Real-Time” The Master Variable: Time Variable Triggers Combined with Variable Results Chapter 3 Foundations Part 1: The Video Image How Do Computers Display Images? What Defines a Computer Image? Making Real-Time Images: Generative Content Making Real-Time Images: Software and Game Engines Chapter 4 Foundations Part 2: Code and Generative Content Generative Content Video Games The Internet VR and AR Case Study – Blair Neal – Trace Your Road Chapter 5 Foundations Part 3: Broadcast Studio Image Compositing Camera POV Sports Broadcast AR Background Replacement Case Study – Bluman Associates – Dave at The BRITS 2020 The Project and Use of Real-Time The Challenges The Tests, Research or Development to Provide the Solutions The End Goal (Not Modified to Be Successful!) and the Result Chapter 6 Real-Time Content Generation Applications Custom Code Processing Effects Engines Media Server Effects Engines Node Based Game Engines Beyond Computer Graphics Case Study – Electronic Countermeasures – The Foo Fighters 26th Anniversary Tour Chapter 7 A Collision of Technologies What Are All the Tools We Combine? Media Control and Management Projection Mapping and Scenery Tracking Environment Modeling Camera Tracking Content Generation Front Plate and Back Plate Virtual Production Stage Types LED Stages/Volumes LED Stages/XR Stages Modern Green Screen Online Events What Happens Next? Hybrid Audiences Remote Performance AR for Live Events Case Study – Rich & Miyu – AR Livestreamer A TouchDesigner Based Software Solution for AR Applications Section 2 Real-Time Content Entertainment Applications Chapter 8 Content Production The No-Render Advantage Creative Flexibility Live Signal Manipulation Interactive Content Production Environmental Responsiveness Camera Perspective Generated Content Case Study – Fray Studio – Real-Time Update Chapter 9 Augmented Reality Virtual Graphic Imagery Scenic Augmentation Human Tracking and Motion Capture Case Study – All of It Now – Coldplay x BTS Project Timeline Technical Considerations Creative Pipeline – The Creative Process On Set Deployment/Challenges Post-Production Challenges A Volumetric Future Chapter 10 Background Replacement Scenic Backdrops Depth Simulation LED vs Green Screen Case Study – Fernando González Ortiz – The Lion and The Firebird The New Frontier in Student Filmmaking with Virtual Production Chapter 11 Extended Reality and Mixed Reality Front Plate and Back Plate Sync Scenic Extension AR + Depth Sensing Mixed Reality LED vs Green Screen, Part 2 Environmental Immersion Lighting and Reflections Budget Speed Background Cohesion Camera Frustums Background Quality Case Study – Lux Machina – Worlds 2020 Chapter 12 The Future of Real-Time Content Developing Industries Developing Technologies Case Study – Bild Studios – Al Wasl – The Interactive Experience How Bild turned Al Wasl Plaza into the world’s largest interactive installation for Expo City Dubai Al Wasl – The Interactive Experience Phase 1 – Technical Feasibility and Creative Exploration Background The Need for a Technical and Creative Feasibility Study Choice of Real-Time Engine Requirements Splitting Up the Real-Time Rendering Power Previz Phase 2 – The App Interactive Audio Setup Phase 3 – Creative Development Phase 4 – Onsite Installation Thoughts by David Bajt, Co-Founder of Bild and Principal Consultant for Al Wasl Expo Projects Thoughts by Expo City Dubai Executive Creative Director, Amna Abulhoul Section 3 Strategies Using Real-Time Content Chapter 13 Team Leadership and Structure The Video Umbrella Content Creation Media Operations System Engineering Broadcast Engineering Video Team Leadership Partner Teams Scenic Design/Art Department/Virtual Art Department Lighting Design Audio Clients of the Video Team Director Producers Case Study – Visual Noise Creative – MTV VMAs 2021 Chapter 14 Inter-Department Communication Crafting Good Collaborative Relationships Case Study – EVOKE Studios – Siemens Hannover Messe Digital 2021 Doing Something First, It Can Be Tricky Plan, Plan, Plan! Creating the Solution The End Result Chapter 15 Production Planning Client Planning Budget Schedule Planning Review Workflow Outline Scenic and Storyboard Review Creative Discussions Team Resources Build and Publish Content Production Workflow Development and Testing, Review and Reiterate Case Study – dandelion & burdock – Johnnie Walker Princes Street Interactive World Map Overview Brief Preparation Process Delivery + Opinion Chapter 16 Obstacles to Success Community Education Inversion of Process Digital Is Perceived as Easy Real-Time Can Be Time Consuming Team Growth and Wellness Case Study – Savages – Practice Makes Process A Case for Intentional Practice Chapter 17 Creative First When Technology Leads Design Discussion What Is the Creative Goal? Technology Serves Creative Intent Matt Swoboda – Founder – Notch Ashraf Nehru – Founder – Disguise Greg Hermanovic – Founder and President – Derivative Francis Maes – Founder – SMODE Tech Emric and Martin – Co-Founders – VYV Case Study – Luke Halls Studio – Forest of Us Appendices Appendix A: Real-Time Content Production Tools and Capabilities Appendix B: VFX Map for Real-Time Content Appendix C: Glossary Index