Bringing Stage Events with Real Actors to Your Home in VR: A Low Complexity Solution for Immersive Media Streaming

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–21
Abstract: In this paper we demonstrate a novel method how to integrate stereo content of a live act into a virtual reality (VR) environment. This opens a new level of immersion, when watching the recording of a stage event using a head mounted display. In contrast to existing solutions, we offer proper depth perception, allow for a low complexity recording and distribution and still avoid any motion sickness caused by objects being close to the observer. To this end, we separate the digitization of the event location from the recording of the actor's performance. As a main contribution of the paper, we show how to achieve high visual quality when compositing the stereo recording showing the actors with the reconstructed 3D-model of the event location. To do so, we have created a tool that allows to simulate the 3D perception of the stereo video depending on the selected stereo parameters such as inter-camera distance and focal length. By these means, we can ensure a smooth transition between the stereo video and the 3D model of the event location, leading to superior visual quality. Numerous lessons learned conclude our paper showing how we overcame practical problems encountered when recording a real theater play.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3010
ISBN: [object Object]
Link: https://doi.org/10.5594/MOO/3010
Author(s): Joachim Keinert
Tobias Jaschke
Hendrik Hawer
Andree Josef
Frank Dittrich
Siegfried Fossel
Keyword(s): Stereo, Parallax, Actor, Live Act, Stage, Event Location, 3d-reconstruction, Blender, Unity, Virtual Reality, Head Mounted Display, Broadcast, 360° Video, Motion Capture, Photogrammetry, LiDAR, Focal Length, Time & Sync
Copyright: © 2024 SMPTE

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3010.json

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Joachim Keinert, Tobias Jaschke, Hendrik Hawer, Andree Josef, Frank Dittrich, and Siegfried Fossel; Bringing Stage Events with Real Actors to Your Home in VR: A Low Complexity Solution for Immersive Media Streaming, MTS 2024, Article 10 (pp. 1 to 21); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3010

Snippet:

Joachim Keinert, Tobias Jaschke, Hendrik Hawer, Andree Josef, Frank Dittrich, and Siegfried Fossel; Bringing Stage Events with Real Actors to Your Home in VR: A Low Complexity Solution for Immersive Media Streaming, MTS 2024, Article 10 (pp. 1 to 21); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3010

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Joachim Keinert, Tobias Jaschke, Hendrik Hawer, Andree Josef, Frank Dittrich, and Siegfried Fossel; <cite>Bringing Stage Events with Real Actors to Your Home in VR: A Low Complexity Solution for Immersive Media Streaming</cite>, MTS 2024, Article 10 (pp. 1 to 21); SMPTE, 2024, ISBN: [object Object]. Available at <a href="https://doi.org/10.5594/MOO/3010" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3010</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Joachim Keinert, Tobias Jaschke, Hendrik Hawer, Andree Josef, Frank Dittrich, and Siegfried Fossel; <cite id="bib-10-5594-moo-3010">Bringing Stage Events with Real Actors to Your Home in VR: A Low Complexity Solution for Immersive Media Streaming</cite>, MTS 2024, Article 10 (pp. 1 to 21); SMPTE, 2024, ISBN: [object Object]
<span class="doi">10.5594/MOO/3010</span>
</li>