Virtual Production
Video production is changing rapidly. A plethora of formats which can be played back on a variety of devices dominate today’s digital space. In addition, the implementation of extended reality technologies (augmented reality, virtual reality, mixed reality) has made immersive content accessible to users via smartphone or tablet, opening up a space for almost infinite creative applications.
For some years now, the topic of immersion has also been increasingly explored in the production processes of film and video – virtual production.
The combination of state-of-the-art game technology and film has already made it possible to produce films in immersive studios for several years. Now this technology is mature and ready for use, experts report. What virtual production is and what impact it will have on film and video production in the future – and accordingly on brands and their marketing measures – is the subject of this insight.
What is virtual production?
Virtual production is a collective term that describes many different digital production processes. These include AR, VR, motion capture and volumetric video (more on this shortly). What we have recently become familiar with from productions such as “The Mandalorian” (2020) and what is commonly referred to as virtual production describes the possibility of producing films in an immersive studio. The entire production of a film is made possible by the combination of reality – i.e. actors and actresses, props – and digital backdrops that are generated in real time in game engines.
What does a virtual studio look like?
In principle, a virtual studio consists of an LED surface that runs in a semicircle or elliptical shape around the set and extends over the ceiling. This surface is covered with a digital backdrop, e.g. a mountain landscape with a sky above it. The possibilities for design are, as you might expect, endless. An immersive space is created with which the actors and crew can interact. For example, an actress can look at a mountain in the far distance without just imagining it, as previously on a set with a greenscreen. In addition, it is possible to experiment with the scenery: landscapes can be changed in real time or colour moods can be adjusted.
On the set of “The Mandalorian”. Film crew in the immersive studio of ILM StageCraft, also called “The Volume”.
Technological developments in the gaming sector make this production process possible in the first place. To achieve a perspective match between the backdrop and the shooting angle, the camera movement in the room is tracked. The room is measured volumetrically beforehand – volumetry refers to a technical measuring procedure in which a room is measured three-dimensionally. The computer can then track all the elements in front of the camera lens and place them in a 3D environment. The game engine then renders the 3D environment in real time – currently, the Unreal Engine is probably the most powerful. The Meta Festival initiated by DEPT® and Journee is also based on the Unreal Engine.
Since the backdrop is already part of the production, the post-production is brought forward and integrated into the pre-production. In some cases, post-production is not necessary at all, since the backdrop exists for every scene and the light emitted by the LED surface already creates the desired colour world. The green screen can still be used to achieve certain effects. It is then simply projected over the LED surface as well.
A selection of films made on virtual sets: “The Batman” (2022), “The Mandalorian” (2020), “Ripple Effect” (2020), “Lion King” (2019), “First Man” (2018). Another innovation in the field of virtual production is volumetric video.
What is volumetric video?
Volumetric video is produced in a special volumetric studio and refers to a video where pixels have not two but three spatial coordinates – giving it volume. In a volumetric studio, cameras are installed so that the object is recorded from all perspectives. Through this recording process, the object can later be viewed from any perspective, similar to gaming, with the difference that the degree of representation is much more realistic.
This technology is particularly interesting for use in the metaverse, e.g. for the creation of photorealistic avatars. A film in which volumetric video was used is, for example, “Matrix Resurrections” (2021). Many shots from the film were created in the Babelsberg studio Volucap.
The Volucap Studio in Babelsberg.
Volumetric video has the further advantage that the camera perspective can be changed in post-production and is not limited by the camera movement as before. This is particularly interesting for interactive films in which the viewers can explore the space themselves.
The possibilities of virtual production grant filmmakers unprecedented control over all elements and brings great flexibility. For example, the film crew is no longer dependent on local weather conditions and can avoid long journeys to locations. Scenes can be replicated afterwards, as lighting conditions and backdrops can be recreated exactly.
Virtual production has also accelerated production processes tremendously. This is reflected above all in production costs, which means that virtual production is increasingly becoming the focus of producers.
What do these developments mean for future film and video production?
The transition to digital production will change the professional field from the ground up; for example, think of production designers who will no longer build physical sets but digital ones in the future. Art departments will become virtual art departments or hybrids. Virtual set objects will become reusable and monetisable, e.g. in the form of NFTs.
There are several arguments in favour of entering virtual production: maximum creative control, flexibility, sustainability – the elimination of building huge sets on locations, transporting film crews and the associated logistics – and correspondingly lower costs.
Will virtual production replace conventional production? Not quite, because it doesn’t make sense in every case. It is therefore worthwhile to subject a project to a proof-of-concept at the beginning to determine which production method is best suited. Nevertheless, one can follow this development with interest.
DEPT® has experts on hand to advise and support you throughout all phases of production. Further information is available on our Production service page. We look forward to working with you!
