Advanced Volumetric Capture and Processing

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 128, No. 5, pp. 18–24
Abstract: Volumetric video is regarded worldwide as the next important development in media production, especially in the context of rapidly evolving virtual and augmented reality markets where volumetric video is becoming a key technology. Fraunhofer Heinrich Hertz Institute (HHI) has developed a novel technology for volumetric video. The 3D Human Body Reconstruction (3DHBR) technology captures real persons with our novel volumetric capture system and creates naturally moving dynamic 3D models, which can then be observed from arbitrary viewpoints in a virtual or augmented reality scene. The capture system consists of an integrated multicamera and lighting system for a full 360° acquisition. A cylindrical studio has been developed with a diameter of 6 m and consists of 32 20-MPixel cameras and 120 light-emitting-diode (LED) panels that allow for an arbitrary lit background. Hence, diffuse lighting and automatic keying are supported. The avoidance of green screen and provision of diffuse lighting offers the best possible conditions for relighting of the dynamic 3D models afterward at the design stage of the virtual reality (VR) experience. In contrast to classical character animation, facial expressions and moving clothes are reconstructed at high geometrical detail and texture quality. The complete workflow is fully automatic, requires about 12 hr/min of mesh sequence, and provides a high level of quality for immediate integration in virtual scenes. Meanwhile, a second, professional studio has been built up on the film campus of Potsdam Babelsberg. This studio is operated by VoluCap GmbH, a joint venture between Studio Babelsberg, ARRI, UFA, Interlake, and Fraunhofer Heinrich Hertz Institute (HHI).
Publication Date: 2019-06-01
DOI: 10.5594/JMI.2019.2906835
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2019.2906835
Author(s): Oliver Schreer
Ingo Feldmann
Thomas Ebner
Sylvain Renault
Christian Weissig
Danny Tatzelt
Peter Kauff
Keyword(s): Augmented Reality, Computer Vision, Meshes, Virtual Reality, Volumetric Video
Copyright: © 2019 SMPTE

Bibliographic Reference(s)

W. Waizenegger et al., “Real-Time Patch Sweeping for High-Quality Depth Estimation in 3D Videoconferencing Applications,” Proc. SPIE Conference on Real-Time Image and Video Processing, San Francisco, CA, 2011. EXTERNAL
N. Robertini, D. Casas, E. de Aguiar, and C. Theobalt, “Multi-View Performance Capture of Surface Details,” Int. J. Comput. Vision, Online, Jan. 2017. EXTERNAL
M. Garland and P. S. Heckbert, “Surface Simplification Using Quadric Error Metrics,” Proc. 24th Ann. Conf. on Comp. Graph. and Inter. Techniques SIGGRAPH ‘97, pp. 209–216, New York, NY, 1997. EXTERNAL
D. Vlasic et al., “Dynamic Shape Capture Using Multi-View Photometric Stereo,” ACM Trans. Graph., 28(5):174, Dec. 2009. EXTERNAL
M. Kazhdan and H. Hoppe, “Screened Poisson Surface Reconstruction,” ACM Trans. Graph., 32(3), June 2013. EXTERNAL
Feldmann, “Visibility-Driven Patch Group Generation,” presented at the IEEE Int. Conf. 3D Imaging, Liege, Belgium, Dec. 2014. EXTERNAL
8i volumetric Video Studio, 2019. [Online]. Available: https://8i.com/ EXTERNAL
4D View Solutions. [Online]. Available: http://www.4dviews.com EXTERNAL
T. Ebner, I. Feldmann, and S. O. Schreer. “46-2: Distinguished Paper: Dynamic Real World Objects in Augmented and Virtual Reality Applications,” SID Symposium Digest of Technical Papers, 48 (1): 673–676, May 2017. EXTERNAL
S. Ebel, W. Waizenegger, M. Reinhardt, O. Schreer, and EXTERNAL
W. Waizenegger, I. Feldmann, O. Schreer, P. Kauff, and P. Eisert, “Real-Time 3D Body Reconstruction for Immersive TV,” presented at the 23rd Int. Conf. Image Processing, Pheonix, AZ, Sept. 2016. EXTERNAL
Unreal Engine, 2019. [Online]. Available: https://www.unrea-lengine.com/en-US/ EXTERNAL
V. Leroy, J.-S. Franco, and E. Boyer, “Multi-View Dynamic Shape Refinement Using Local Temporal Integration,” Proc. IEEE International Conference on Computer Vision, Venice, Italy, Oct. 2017. EXTERNAL
Unity Company Website, 2019. [Online]. Available: https://unity.com/ EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2019.2906835.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, and Peter Kauff; Advanced Volumetric Capture and Processing, SMPTE Motion Imaging Journal ( Volume: 128, Issue: 5, June 2019); SMPTE, 2019. Available at https://doi.org/10.5594/JMI.2019.2906835

Snippet:

Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, and Peter Kauff; Advanced Volumetric Capture and Processing, SMPTE Motion Imaging Journal ( Volume: 128, Issue: 5, June 2019); SMPTE, 2019. Available at https://doi.org/10.5594/JMI.2019.2906835

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, and Peter Kauff; <cite>Advanced Volumetric Capture and Processing</cite>, SMPTE Motion Imaging Journal ( Volume: 128, Issue: 5, June 2019); SMPTE, 2019. Available at <a href="https://doi.org/10.5594/JMI.2019.2906835" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2019.2906835</a></span>

SMPTE's HTML Pub

Preview:

Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, and Peter Kauff; Advanced Volumetric Capture and Processing, SMPTE Motion Imaging Journal ( Volume: 128, Issue: 5, June 2019); SMPTE, 2019
doi: 10.5594/JMI.2019.2906835
url: https://doi.org/10.5594/JMI.2019.2906835

Snippet:

<li>
Oliver Schreer, Ingo Feldmann, Thomas Ebner, Sylvain Renault, Christian Weissig, Danny Tatzelt, and Peter Kauff; <cite id="bib-10-5594-jmi-2019-2906835">Advanced Volumetric Capture and Processing</cite>, SMPTE Motion Imaging Journal ( Volume: 128, Issue: 5, June 2019); SMPTE, 2019
<span class="doi">10.5594/JMI.2019.2906835</span>
</li>