Demystifying the use of Compression in Virtual Production

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–19
Abstract: Virtual Production (VP) technologies have continued to improve the flexibility of on-set filming and enhance the live concert experience. The core technology of VP relies on high-resolution, high-brightness LED panels to playback/render video content. There are a number of technical challenges to effective deployment e.g. image tile synchronisation across the panels, cross panel colour balancing and compensating for colour fluctuations due to changes in camera angles. Given the complexity and potential quality degradation, the industry prefers “pristine” or lossless compressed source material for displays, which requires significant storage and bandwidth. Modern lossy compression standards like AV1 or H.265 could maintain the same quality at significantly lower bitrates and resource demands. There is yet no agreed methodology for assessing the impact of these standards on quality when the VP scene is recorded in-camera. We present a methodology to assess this impact by comparing lossless and lossy compressed footage displayed through VP screens and recorded in-camera. We assess the quality impact of HAP/NotchLC/Daniel2 and AV1/HEVC/H.264 compression bitrates from 2 Mb/s to 2000 Mb/s with various GOP sizes. Several perceptual quality metrics are then used to automatically evaluate in-camera picture quality, referencing the original uncompressed source content through the LED wall. Our results show that we can achieve the same quality with hybrid codecs as with intermediate encoders at orders of magnitude less bitrate and storage requirements.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3011
ISBN: Electronic: 978-1-61482-965-2
Link: https://doi.org/10.5594/MOO/3011
Author(s): Anil Kokaram
Vibhoothi Vibhoothi
Julien Zouein
Francois Pitie
Christopher Nash
James Bentley
Philip Coulam-Jones
Keyword(s): Virtual Production, Compression, HEVC, H.264, AV1, HAP, NotchLC, Daniel2
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

G. W. Perkins and S. Echeverry, “Virtual Production in Action: A Creative Implementation of Expanded Cinematography and Narratives,” in ACM SIGGRAPH Posters, 2022. EXTERNAL
T. Zhang, “From digital visual effects to emerging in-camera visual effects: investigating the change of workflow, occupational roles and common challenges in Southeast Asian and East Asian countries.,” Nanyang Technological University, 2024. EXTERNAL
Disguise, Welcome to the New Era of Film: Five ways virtual production is revolutionising filmmaking, Disguise, 2023. EXTERNAL
N. Kadner, The Virtual Production Field Guide Volume 1, Epic Games, 2019. EXTERNAL
P. Debevec and C. LeGendre, “HDR Lighting Dilation for Dynamic Range Reduction on Virtual Production Stages.,” in ACM SIGGRAPH 2022 Posters, 2022. EXTERNAL
R. Hendricks, “Rapid Industry Solutions (RiS) Initiative, On-Set Virtual Production (OSVP), and Open Services Alliance (OSA),” SMPTE Motion Imaging Journal, vol. 8, no. 132, pp. 78–80, 2023. EXTERNAL
B. Desowitz, “Reverse Engineering 'Gravity’,” Animation World Network, 4 October 2013. [Online]. Available: https://www.awn.com/vfxworld/reverse-engineering-gravity. [Accessed 03 August 2024]. EXTERNAL
S. Dalkian, S. Miglio and others, “nDisplay Technology: Limitless scaling of real-time content,” Technical Whitepaper, pp. 1–24, 2019. EXTERNAL
Disguise, “Eight benefits of virtual production,” Disguise, April 2021. [Online]. Available: https://www.disguise.one/en/insights/blog/eight-benefits-of-virtual-production/. [Accessed 05 August 2024]. EXTERNAL
M. Kavakli and C. Cremona, “The virtual production studio concept-an emerging game changer in filmmaking,” IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 29–37, 2022. EXTERNAL
F. Pires, R. Silva and R. Raposo, “A survey on virtual production and the future of compositing technologies,” Avanca Cinema Journal, vol. 21, no. 692–9, 2022. EXTERNAL
Tom Whittock, “The Key to Superior Content Rendering,” Disguise, 12 2023. [Online]. Available: https://www.disguise.one/sites/default/files/2023–12/Renderstream_Whitepaper.pdf. [Accessed 05 August 2024]. EXTERNAL
M. Henrique and G. Sullivan, “YCoCg-R: A color space with RGB reversibility and low dynamic range,” ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q6, 2003. EXTERNAL
Cinegy, “Cinegy Daniel2 GPU Codec Enters Public Beta,” Cinegy, 6 June 2017. [Online]. Available: https://www.cinegy.com/news/cinegy-daniel2-gpu-codec-enters-public-beta/. [Accessed 06 August 2024]. EXTERNAL
Kostya, “A cursory glance at Daniel codecs,” 10 July 2022. [Online]. Available: https://codecs.multimedia.cx/2022/07/a-cursory-glance-at-daniel-codecs/. [Accessed 06 August 2024]. EXTERNAL
N. Porrmann, “10bit Workflow Viability + Codec Comparison,” Dandelion-burdock, June 2019. [Online]. Available: https://dandelion-burdock.com/articles/10bit-workflow-viability. [Accessed 6 August 2024]. EXTERNAL
K. Seshadrinathan, R. Soundararajan, A. C. Bovik and L. K. Cormack, “Study of subjective and objective quality assessment of video,” IEEE transactions on Image Processing, vol. 19, no. 6, pp. 1427–1441, 2010. EXTERNAL
V. Vibhoothi, A. Katsenou, F. Pitie, K. Domijan and A. Kokaram, “Subjective Assessment of the Impact of a Content Adaptive Optimiser for Compressing 4K HDR Content With AV1,” in 2023 IEEE International Conference on Image Processing (ICIP), Kuala Lumpur, Malaysia,,2023. EXTERNAL
K. Murray and V. Simion, Color Management in Virtual Production, vol. 20, Los Angeles: ACM SIGGRAPH, 2023. EXTERNAL
J. Y. Lin, E. C.-H. Wu, C.-C. J. Kuo, P. L. Callet, T. Goodall and A. Aaron, Toward A Practical Perceptual Video Quality Metric, Los Gatos: Netflix Technical Blog, 2016. EXTERNAL
Z. Li, C. Bampis, J. Novak, A. Aaron, K. Swanson, A. Moorthy and J. De Cock, “VMAF: The Journey Continues,” Netflix, 25 October 2018. [Online]. Available: https://netflixtechblog.com/vmaf-the-journey-continues-44b51eeged12. [Accessed 11 August 2024]. EXTERNAL
R. K. Mantiuk, P. Hanji, M. Ashraf, Y. Asano and A. Chapiro, “ColorVideoVDP: A visual difference predictor for image, video and display distortions,” ACM Trans. Graph, vol. 43, no. 4, p. 20, 2024. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3011.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Anil Kokaram, Vibhoothi Vibhoothi, Julien Zouein, Francois Pitie, Christopher Nash, James Bentley, and Philip Coulam-Jones; Demystifying the use of Compression in Virtual Production, MTS 2024, Article 11 (pp. 1 to 19); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3011

Snippet:

Anil Kokaram, Vibhoothi Vibhoothi, Julien Zouein, Francois Pitie, Christopher Nash, James Bentley, and Philip Coulam-Jones; Demystifying the use of Compression in Virtual Production, MTS 2024, Article 11 (pp. 1 to 19); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3011

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Anil Kokaram, Vibhoothi Vibhoothi, Julien Zouein, Francois Pitie, Christopher Nash, James Bentley, and Philip Coulam-Jones; <cite>Demystifying the use of Compression in Virtual Production</cite>, MTS 2024, Article 11 (pp. 1 to 19); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at <a href="https://doi.org/10.5594/MOO/3011" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3011</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Anil Kokaram, Vibhoothi Vibhoothi, Julien Zouein, Francois Pitie, Christopher Nash, James Bentley, and Philip Coulam-Jones; <cite id="bib-10-5594-moo-3011">Demystifying the use of Compression in Virtual Production</cite>, MTS 2024, Article 11 (pp. 1 to 19); SMPTE, 2024, ISBN: 978-1-61482-965-2
<span class="doi">10.5594/MOO/3011</span>
</li>