MPEG-I Standards for Volumetric Video and a Real-Time Implementation

Metadata

Publisher: SMPTE
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Motion Imaging Jour.
Volume: 133, No. 4, pp. 66–75
Abstract: This paper presents a real-time implementation of a platform jointly developed by InterDigital and Philips that showcases use cases leveraging the MPEG volumetric (MPEG-I V3C) and 2D video standards Versatile Video Coding [VVC], High Efficiency Video Coding [HEVC]. We will detail how our platform enables interoperability within existing and emerging extended reality (XR) ecosystems, including the acquisition, streaming, and real-time interactive playback of volumetric video on current and future client devices for use in applications like telelearning, free-viewpoint sports replays, and 3D telepresence in connected ecosystems like the metaverse. MPEG's Visual Volumetric Video-based Coding (V3C) standard is an extensive framework for coding volumetric video, from dynamic point clouds (V-PCC) to multiview plus depth and multiplane image representations (MIV). It offers a single bitstream structure with a uniform bridge to systems-level standards. The V3C carriage standard defines how volumetric content can be stored, transported, and delivered to the end user. It repurposes existing 2D video hardware decoding capabilities and graphic processing units (GPUs) to decode and render volumetric video.
Publication Date: 2024-07-01
DOI: 10.5594/JMI.2024/IXTK5404
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2024/IXTK5404
Author(s): Celine Guede
Patrick Fontaine
Julien Mulard
Bertrand Leroy
Cyril Quinquis
Remy Gendrot
Srinivas Gudumasu
Valerie Allie
Bart Kroon
Bart Sonneveldt
Ronald Schimanofsky
Keyword(s): QoE, Immersive Video, Playback-Side Context, Image Depth and Parallax
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

V-PCC Performance Evaluation of the First MPEG Point Codec [Active]
ISO.23090-5 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-5, “Visual Volumetric Video-Based Coding (V3C) and Video-Based Point Cloud Compression (V-PCC).” EXTERNAL
ISO.23090-10 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-10, “Carriage of Visual Volumetric Video-Based Coding Data.” EXTERNAL
ISO.23090-12 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-12, Information technology—Coded Representation of Immersive Media—Part 12: MPEG immersive video. EXTERNAL
ISO.23090-14 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-14, Information technology—Coded representation of immersive media—Part 14: Scene description. EXTERNAL
International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) JTC1/SC29 WG11, “Use Cases for Point Cloud Compression,” Genova, Switzerland, w16331, Jun. 2016. EXTERNAL
Bart Kroon MIV Tutorial, IEE VCIP 2021. [Online]. Available: https://mpeg-miv.org/wp-content/uploads/2021/12/1-MIV-tutorial-part-1.pdf. EXTERNAL
International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) JTC1/SC29 WG11, Call for proposals for point cloud compression v2, N16763, 2017. EXTERNAL
International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) JTC1/SC29 WG11, Online, April 2021, MDS20352, V-PCC codec description. EXTERNAL
J. M. Boyce, “MPEG Immersive Video Coding Standard,” in Proc. IEEE, 109 (9): 1521–1536, Sept. 2021, doi: 10.1109/JPROC.2021.3062590. EXTERNAL
Six Degrees of Freedom (6DoF). [Online]. Available: https://en.wikipedia.org/wikiSix_degrees_of_freedom. EXTERNAL
Test Model for Category 2 (TMC2) release 22.0 for MPEG V-PCC, MPEG 142, April 2023. [Online]. Available: https://dms.mpeg.expert/doc_end_user/documents/142_Antalya/wg11/MDS22732_WG07_N00572.zip. Public software is also available at MPEG-I3DG/TMC2Github. EXTERNAL
Test Model Version 14 for MPEG Immersive Video, MPEG 139, Jul. 2022. [Online]: Available: https://dms.mpeg.expert/doc_end_user/documents/139_OnLine/wg11/MDS21853_WG04_N00242.zip. Public software is also available at https://gitlab.com/mpeg-i-visual/tmiv. EXTERNAL
FFmpeg. [Online]. Available: https://ffmpeg.org/, v5.1.1. EXTERNAL
OpenVVC, FFmpeg fork with OpenVVC lib support developed by INSA/IETR. [Online]. Available: https://github.com/OpenVVC/FFmpeg. EXTERNAL
NVidia Video Codec SDK. [Online]. Available: https://developer.nvidia.com/video-co-dec-sdk. EXTERNAL
Android MediaCodec. [Online]. Available: https://developer.android.com/reference/android/media/MediaCodec. EXTERNAL
OpenVVC, real-time software decoder developed by INSA/IETR. [Online]. Available: https://github.com/OpenVVC/OpenVVC. EXTERNAL
Unity v2020.3.22f1. [Online]. Available: https://unity.com/. EXTERNAL
Unity Plugins. [Online]. Available: https://docs.unity3d.com/Manual/Plugins.html. EXTERNAL
Unity UI Toolkits. [Online]. Available: https://docs.unity3d.com/Manual/UIToolkits.html. EXTERNAL
Unreal Engine. [Online]. Available: https://www.unrealengine.com/. EXTERNAL
XD Productions. [Online]. Available: https://www.xdprod.com/. EXTERNAL
International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) JTC1/SC29 WG11, “Common Test Conditions for PCC,” Alpbach, Austria, w19324, Apr. 2020. EXTERNAL
MPEG Immersive Video. [Online] Available: https://mpeg-miv.org. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2024-IXTK5404.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Celine Guede, Patrick Fontaine, Julien Mulard, Bertrand Leroy, Cyril Quinquis, Remy Gendrot, Srinivas Gudumasu, Valerie Allie, Bart Kroon, Bart Sonneveldt, and Ronald Schimanofsky; MPEG-I Standards for Volumetric Video and a Real-Time Implementation, MIJ 2024, Volume 133, Number 4 (pp. 66 to 75); SMPTE, 2024. Available at https://doi.org/10.5594/JMI.2024/IXTK5404

Snippet:

Celine Guede, Patrick Fontaine, Julien Mulard, Bertrand Leroy, Cyril Quinquis, Remy Gendrot, Srinivas Gudumasu, Valerie Allie, Bart Kroon, Bart Sonneveldt, and Ronald Schimanofsky; MPEG-I Standards for Volumetric Video and a Real-Time Implementation, MIJ 2024, Volume 133, Number 4 (pp. 66 to 75); SMPTE, 2024. Available at https://doi.org/10.5594/JMI.2024/IXTK5404

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Celine Guede, Patrick Fontaine, Julien Mulard, Bertrand Leroy, Cyril Quinquis, Remy Gendrot, Srinivas Gudumasu, Valerie Allie, Bart Kroon, Bart Sonneveldt, and Ronald Schimanofsky; <cite>MPEG-I Standards for Volumetric Video and a Real-Time Implementation</cite>, MIJ 2024, Volume 133, Number 4 (pp. 66 to 75); SMPTE, 2024. Available at <a href="https://doi.org/10.5594/JMI.2024/IXTK5404" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2024/IXTK5404</a></span>

SMPTE's HTML Pub

Preview:

Celine Guede, Patrick Fontaine, Julien Mulard, Bertrand Leroy, Cyril Quinquis, Remy Gendrot, Srinivas Gudumasu, Valerie Allie, Bart Kroon, Bart Sonneveldt, and Ronald Schimanofsky; MPEG-I Standards for Volumetric Video and a Real-Time Implementation, MIJ 2024, Volume 133, Number 4 (pp. 66 to 75); SMPTE, 2024
doi: 10.5594/JMI.2024/IXTK5404
url: https://doi.org/10.5594/JMI.2024/IXTK5404

Snippet:

<li>
Celine Guede, Patrick Fontaine, Julien Mulard, Bertrand Leroy, Cyril Quinquis, Remy Gendrot, Srinivas Gudumasu, Valerie Allie, Bart Kroon, Bart Sonneveldt, and Ronald Schimanofsky; <cite id="bib-10-5594-jmi-2024-ixtk5404">MPEG-I Standards for Volumetric Video and a Real-Time Implementation</cite>, MIJ 2024, Volume 133, Number 4 (pp. 66 to 75); SMPTE, 2024
<span class="doi">10.5594/JMI.2024/IXTK5404</span>
</li>