What is Virtual Production?
Virtual production is a filmmaking technique that combines real-world scenes with computer-generated imagery to create photorealistic environments in real time. Advances in graphics processing unit (GPU) and game engine technologies have made real-time photorealistic visual effects (VFX) a reality. The emergence of real-time photorealistic VFX has sparked a revolution in the film and television industry. With virtual production, the physical and digital worlds can now interact seamlessly with photorealistic quality.
By incorporating game engine technology and fully immersive LED screens into the creative workflow, virtual production enhances the efficiency of the creative process, leading to a more seamless screen experience. At a high level, virtual production allows previously siloed creative teams to collaborate in real time and make decisions faster, as each team can see what the final shot will look like during the actual filming.
Disruptive Technology in Film and Television
Disruptive technology refers to innovations that significantly change the way consumers, industries, and businesses operate. For the film and television industry, this began with the transition from silent films to talkies, then from black-and-white to color, followed by television, home video tapes, DVDs, and more recently, streaming services.
Over the years, the methods used to produce films and TV shows have undergone significant technological transformations. The major shift discussed in the remainder of this article is the transition to modern visual effects, pioneered by films like Jurassic Park and The Terminator. Other milestone VFX films include The Matrix, The Lord of the Rings, Avatar, and Gravity. Film enthusiasts are encouraged to share their thoughts on which movies were pioneers or milestones in modern VFX.
Traditionally, film and TV production has been divided into three stages: pre-production, production, and post-production. In the past, visual effects were created during post-production, but emerging virtual production methods have moved much of the VFX process into the pre-production and production stages, with post-production reserved for specific shots and post-shoot fixes.
LED Screens in Creative Workflows
Virtual production integrates multiple technologies into a single, cohesive system. Traditionally unrelated fields are converging, leading to new partnerships, processes, technologies, and more. Virtual production is still in its early adoption stage, and many are working to understand it.
Anyone who has researched this topic may have come across Mike Seymour’s articles on FX Guide, The Art of Virtual Production on LED Walls, Part One and Part Two. These articles provide insights into the making of The Mandalorian, which was largely shot on direct-view LED screens. Seymour outlines lessons learned during the production of The Mandalorian and how virtual production is changing creative workflows. The second part reviews the technical aspects and challenges faced when implementing in-camera VFX.
Sharing this level of thought leadership drives film and TV producers’ understanding of the latest technological advances. With several films and TV shows successfully utilizing real-time VFX, the race to adopt the latest workflows is on. The further adoption of virtual production has been partly driven by the pandemic, which pushed the world towards remote work and required all businesses and organizations to rethink how they operate.
Designing LED Screens for Virtual Production
Given the range of technologies required for virtual production, determining each technology’s performance and understanding the actual meaning of specifications requires collaboration among experts from various fields. This brings us to the true purpose of this article, writing from the perspective of an industry-leading direct-view LED manufacturer on designing LED screens for virtual production.
LED Screen Configuration
The configuration and curvature of LED volumes largely depend on how the virtual background will be captured and how the camera will move during the shoot. Will the volume be used for broadcasting and live streaming? If so, will the camera be shooting from a fixed angle or panning around a focal point? Or will the virtual scene be used for full-motion video? If so, how will personnel and materials be captured within the volume? These types of considerations help LED volume designers determine the appropriate screen size, whether the screen should be flat or curved, and the requirements for angles, ceilings, and/or floors. Key factors to manage include providing a large enough canvas to allow for a complete viewing cone while minimizing color shift caused by the viewing angle of the LED panels that make up the screen.
Pixel Pitch
Moiré patterns can be a major issue when filming LED screens. Choosing the correct pixel pitch is the best way to eliminate moiré patterns. If you’re unfamiliar with pixel pitch, you can learn more about it here. Moiré patterns are caused by high-frequency interference patterns resulting from the camera picking up individual pixels on the LED screen. In virtual production, the relationship between pixel pitch and viewing distance relates not only to the camera’s position but also to the nearest point of focus for all scenes. Moiré effects occur when the focus is within the optimal viewing distance for the corresponding pixel pitch. Depth-of-field adjustments can further reduce moiré effects by slightly softening the background. As a rule of thumb, multiply the pixel pitch by ten to get the optimal viewing distance in feet.
Refresh Rate and Flicker
Flicker when filming monitors or LED screens is caused by a mismatch between the display’s refresh rate and the camera’s frame rate. LED screens require a high refresh rate of 3840Hz, which helps eliminate screen flicker and is absolutely necessary for virtual production applications. Ensuring that the LED screen has a high refresh rate is the first step in avoiding screen flicker when filming, aligning the camera’s shutter speed with the refresh rate is the final solution to the problem.
Brightness
For LED screens used in off-camera applications, higher brightness is generally considered better. However, for virtual production, LED screens are often too bright, so brightness is significantly reduced. When the brightness of an LED screen is reduced, color performance is affected. With fewer intensity levels available for each color, grayscale is reduced. Ensuring that the LED screen’s maximum brightness aligns with the maximum light output required for adequate lighting within the LED volume can minimize the extent to which the screen’s brightness needs to be reduced and minimize the loss of color performance.
Color Space, Grayscale, and Contrast
The color performance of an LED screen is composed of three main components: color space, grayscale, and contrast. Color space and grayscale play significant roles in virtual production applications, while contrast is less important.
Color space refers to the specific organization of colors that the screen can achieve. Manufacturers should consider the required color space in advance, as LED screens can be designed to have different color spaces if necessary.
Grayscale, measured in bits, indicates how many intensity levels are available for each color. Generally, the higher the bit depth, the more colors available, resulting in smoother color transitions and eliminating banding. For virtual production LED screens, a grayscale of 12 bits or higher is recommended.
Contrast refers to the difference between the brightest white and the darkest black. In theory, it allows viewers to distinguish content in the image regardless of brightness. However, this specification is often misunderstood. Higher brightness LED screens have higher contrast. Another extreme is the fill factor, using smaller (usually cheaper) LEDs can increase the black in the display, thus improving contrast. While contrast is important, it’s crucial to understand the factors that determine contrast.
Visualization of Setup
Effectively designing LED volumes for space and production is the first step to successfully implementing LED technology for virtual production. Given the custom nature of LED screens, virtually constructing the LED volume in a 3D world is the most effective way to plan screen size, curves, installation, and viewing distances. This allows producers and engineers to visualize the volume and discuss needs in advance, making informed decisions throughout the process.
Site Preparation
Last but not least, throughout the design process, important site-specific themes, including but not limited to structural, power, data, and ventilation requirements, are considered as the team designs and discusses the LED volume. All these factors need to be properly considered and provided to ensure the correct implementation of the designed LED screen.
Conclusion
Virtual production represents a groundbreaking shift in the filmmaking industry, seamlessly integrating real-world elements with digital environments to create stunning, photorealistic visuals. As the technology continues to evolve, the role of high-quality LED screens becomes increasingly vital. For filmmakers and production teams looking to harness the power of virtual production, choosing the right LED screen provider is crucial.
Hot Electronics stands at the forefront of this innovation, offering industry-leading direct-view LED screens designed specifically for virtual production environments. Our screens are engineered to meet the rigorous demands of modern filmmaking, delivering exceptional color accuracy, brightness, and resolution. With our extensive experience and commitment to excellence, we are well-positioned to support your virtual production needs and help bring your creative vision to life.
For more information on how Hot Electronics can elevate your virtual production, contact us today. Let’s work together to push the boundaries of filmmaking and create extraordinary experiences.
Post time: Sep-03-2024