Novel Stereoscopic Content Production Tools

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 120, No. 4, pp. 29–35
Abstract: Stereoscopic 3D is on the cusp of becoming a mass consumer product. But despite the recent rise of stereoscopic movies and technological advances in the industry, many challenges regarding 3D content creation and display remain unsolved. A fundamental issue is control and modification of the stereoscopic depth of live-action content after it has been recorded, since changing depth effectively amounts to changing the baseline and vergence of the capture rig. With existing post-production tools, the modification of these parameters involves cumbersome and expensive segmentation, dense depth reconstruction, and inpainting. This paper provides an overview of existing technologies for stereoscopic content editing, identify current challenges, and presents a number of recent research results that provide novel solutions to problems such as disparity correction and depth authoring, display adaptation, and 2D- to 3D conversion.
Publication Date: 2011-05-01
DOI: 10.5594/j18041XY
ISSN: Print: 1545-0279
Link: https://doi.org/10.5594/j18041XY
Author(s): Alexander Hornung
bio
Alexander Hornung is a research scientist at Disney Research Zurich, heading the group for imaging and video processing. He is also an adjunct lecturer at ETH Zurich and supervises undergraduate as well as M.S. and Ph.D. students. Before joining Disney he was a Postdoc student in the Computer Graphics Laboratory of Prof. Markus Gross at ETH Zurich. Hornung received a Diplom (2003) and Ph.D. (2008) in computer science at the Computer Graphics Group of Prof. Leif Kobbelt at RWTH Aachen. His research interests are computer graphics and vision. In particular, he is interested in the fields of video and light field processing, stereoscopy, image-based rendering and 3D reconstruction, and 2D animation and games
Aljoscha Smolic
bio
Aljoscha Smolic received a Dr.-Ing. degree from Aachen University of Technology (RWTH) in 2001. He joined Disney Research Zurich in 2009, where he leads the “Advanced Video Technology” group. Before that, he was a scientific project manager at Fraunhofer HHI in Berlin. He conducted research in video processing, video coding, computer vision and computer graphics and published more than 100 referred papers in these fields. Since 2003 Smolic had teaching appointments at TU Berlin, ETH Zurich, Universitat Politécnica de Catalunya, Universidad Politécnica de Madrid, and Universitat de les Illes Balears. He is area editor for Signal Processing: Image Communication and served as guest editor for the Proceedings of the IEEE, IEEE Transactions on CSVT, IEEE Signal Processing magazine and others. He chaired the MPEG ad hoc group on 3DAV pioneering standards for 3D video and served as one of the editors of the Multi-view Video Coding (MVC) standard.
Markus Gross
bio
Markus Gross is a professor of computer science at ETH Zurich, head of the Computer Graphics Laboratory, and the director of Disney Research Zurich. For more than 20 years Gross has been pursuing basic and applied research in computer graphics, image generation and display, geometric modeling, and computer animation. His research interests include point-based graphics, physically-based modeling, immersive displays, and video processing. He has published more than 200 scientific papers and holds various patents on core graphics and visualization technologies. Gross was chair of the papers committee of ACM SIGGRAPH 2005 and is a fellow of the Eurographics Association. He serves on the boards of numerous international research institutions, societies, and governmental organizations. He is a co-founder of Cyfex AG, Novodex AG, LiberoVision AG, and Dybuster AG.
Copyright: © 2011 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

1 Howard I. P. Rogers B. J. , Seeing in Depth , Oxford University Press : New York , 2002 . EXTERNAL
2 Hoffman D. M. Girshick A. R. Akeley K. Banks M. S. , “Vergence-Accommodation Conflicts Hinder Visual Performance and Cause Visual Fatigue,” J. Vision , 8 ( 3 ): 1 – 30 , Mar. 2008 . EXTERNAL
3 Mendiburu B. , 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen , Elsevier : Oxford, U.K. , 2008 . EXTERNAL
4 Zitnick C. L. Kang S. B. Uyttendaele M. Winder S. Szeliski R. , “High-Quality Video View Interpolation Using a Layered Representation,” Proc. ACM SIGGRAPH and ACM Trans. Graphics , Los Angeles , 2004 . EXTERNAL
5 Fehn C. Kauff P. Op de Beeck M. Ernst F. Ijsselsteijn W. Pollefeys M. Vangool L. Ofek E. Sexton I. , “An Evolutionary and Optimised Approach on 3D-TV,” Proc. IBC 2002, International Broadcast Convention , Amsterdam , The Netherlands , 2002 . EXTERNAL
6 Bertalmío M. Sapiro G. Caselles V. Ballester C. , “Image Inpainting,” Proc. of ACM SIGGRAPH , New Orleans , 2000 . EXTERNAL
7 Mueller K. Smolic A. Dix K. Merkle P. Kauff P. Wiegand T. , “View Synthesis for Advanced 3D Video Systems,” EURASIP J. Image. Video Processing , Nov. 2008 . EXTERNAL
8 Hornung A. Kobbelt L. , “Interactive Pixel-Accurate Free Viewpoint Rendering from Images with Silhouette Aware Sampling,” Comput. Graph. Forum , 28 ( 8 ): 2090 – 2103 , June 2009 . EXTERNAL
9 Atzpadin N. Kauff P. Schreer O. , “Stereo Analysis by Hybrid Recursive Matching for Real-Time Immersive Video Conferencing,” IEEE Trans. Circuits Syst. Video Technol ., 14 ( 3 ): 321 – 334 , Mar. 2004 . EXTERNAL
10 Scharstein D. Szeliski R. , “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int. J. Comput. Vision , 47 ( 1 ): 7 – 42 , May 2002 . EXTERNAL
11 Seitz S. M. Curless B. Diebel J. Scharstein D. Szeliski R. , “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” Proc. IEEE Conf. Computer Society Conf. Vision and Pattern Recognition , pp. 519 – 528 , 2006 . EXTERNAL
12 The Middlebury Computer Vision Pages, http://vision.middlebury.edu/stereo/ . EXTERNAL
13 3D4YOU, http://www.3d4you.eu/index.php . EXTERNAL
14 Bartczak B. Koch R. , “Dense Depth Maps from Low Resolution Time-of-Flight Depth and High Resolution Color Views,” Proc. 5th International Symposium on Visual Computing (ISVC '09 ), Las Vegas , Dec. 2009 . EXTERNAL
15 Laurentini A. , “The Visual Hull Concept for Silhouette-Based Image Understanding,” IEEE Trans. Pattern Anal. Mach. Intell ., 16 ( 2 ): 150 – 162 , Feb. 1994 . EXTERNAL
16 Matusik W. Buehler C. Raskar R. Gortler S. J. McMillan L. , “Image-Based Visual Hulls,” Proc. ACM SIGGRAPH 2000 , pp. 369 – 374 , 2000 . EXTERNAL
17 Liang C. Wong K. K. , “3D Reconstruction Using Silhouettes from Unordered Viewpoints,” Image Vision Comp ., 28 ( 4 ): 579 – 589 , Apr. 2010 . EXTERNAL
18 de Aguiar E. Stoll C. Theobalt C. Ahmed N. Seidel H. P. Thrun S. , “Performance Capture from Sparse Multi-View Video,” Proc. ACM SIGGRAPH ‘08 , pp. 1 – 10 , 2008 . EXTERNAL
19 Liu Y.-F. , “A Unified Approach to Image Focus and Defocus Analysis,” Ph.D. Dissertation, State University of New York at Stony Brook, 1998 . EXTERNAL
20 Wei T.-C. , “Three Dimensional Machine Vision Using Image Defocus,” Ph.D. Dissertation, State University of New York at Stony Brook, 1994 . EXTERNAL
21 Zhang R. Tsai P.-S. Cryer J. E. Shah M. , “Shape from Shading: A Survey,” IEEE Trans. Pattern Anal. Mach. Intell ., 21 ( 8 ): 690 – 706 , Aug. 1999 . EXTERNAL
22 Durou J.-D. Falcone M. Sagona M. , “Numerical Methods for Shape-from-Shading: A New Survey with Benchmarks,” Comput. Vision. Image Understanding , 109 ( 1 ): 22 – 43 , Jan. 2008 . EXTERNAL
23 Beardsley P. Torr P. H. S. Zisserman A. , “3D Model Acquisition from Extended Image Sequences,” Proc. of the European Conf. on Computer Vision (ECCV ), pp. 683 – 695 , 1996 . EXTERNAL
24 Jebara T. Azarbayejani A. Pentland A. , “3D structure from 2D motion,” IEEE Signal Process. Mag ., 16 ( 3 ): 66 – 84 , May 1999 . EXTERNAL
25 Hartley R.-I. Zisserman A. , Multiple View Geometry in Computer Vision , 2nd ed. , Cambridge University Press : Cambridge, U.K. , 2004 . EXTERNAL
26 Pollefeys M. Van Gool L. Vergauwen M. Verbiest F. Cornelis K. Tops J. Koch R. , “Visual Modeling with a Hand-Held Camera,” Int. J. Comput. Vision , 59 ( 3 ): 207 – 232 , Sept.–Oct. 2004 . EXTERNAL
27 Hoiem D. Efros A. A. Hebert M. , “Automatic Photo Pop-up,” Proc. ACM SIGGRAPH ‘05 , pp. 577 – 584 , 2005 . EXTERNAL
28 Lee S. Feng D. Gooch B. , “Perception-Based Construction of 3D Models from Line Drawings,” Proc. ACM Symposium on Interactive 3D Graphics and Games , 2008 . EXTERNAL
29 Parodi P. Piccioli G. , “3D Shape Reconstruction by Using Vanishing Points,” IEEE Trans. Pattern Anal. Mach. Intell ., 18 ( 2 ): 211 – 217 , Feb. 1996 . EXTERNAL
30 Bai X. Wang J. Simons D. Sapiro G. , “Video SnapCut: Robust Video Object Cutout Using Localized Classifiers,” ACM Trans. Graph ., 28 ( 3 ), Aug. 2009 . EXTERNAL
31 van den Hengel A. Dick A. R. Thormählen T. Ward B. Torr Philip H. S. , “VideoTrace: Rapid Interactive Scene Modelling from Video,” ACM Trans. Graph ., 26 ( 3 ): 86 , July 2007 . EXTERNAL
32 Guttmann M. Wolf L. Cohen-Or D. , “Semi-Automatic Stereo Extraction from Video Footage,” Proc. IEEE International Conf. Computer Vision (ICCV ), pp. 136 – 142 , 2009 . EXTERNAL
33 Seitz S. Dyer C. , “View morphing,” Proc. SIGGRAPH ‘96 , pp. 21 – 30 , 1996 . EXTERNAL
34 Mahajan D. Huang F.-C. Matusik W. Ramamoorthi R. Belhumeur P. N. , “Moving Gradients: A Path-Based Method for Plausible Image Interpolation,” ACM Trans. Graph ., 23 ( 3 ): 814 – 824 , July 2009 . EXTERNAL
35 Krähenbühl P. Lang M. Hornung A. Gross M. , “A System for Retargeting of Streaming Video,” ACM Trans. Graph ., 28 ( 5 ), Dec. 2009 . EXTERNAL
36 Carroll R. Agrawala M. Agrawala A. , “Optimizing Content-Preserving Projections for Wide-Angle Images,” ACM Trans. Graph ., 28 ( 3 ), Aug. 2009 . EXTERNAL
37 Carroll R. Agrawala M. Agrawala A. , “Image Warps for Artistic Perspective Manipulation,” ACM Trans. Graph ., 29 ( 4 ), July 2010 . EXTERNAL
38 Liu F. Gleicher M. Jin H. Agrawala A. , “Content-Preserving Warps for 3D Video Stabilization,” ACM Trans. Graph ., 28 ( 3 ), Aug. 2009 . EXTERNAL
39 Lang M. Hornung A. Wang O. Poulakos S. Smolic A. Gross M. , “Nonlinear Disparity Mapping for Stereoscopic 3D,” ACM Trans. Graph ., ( 29 ) 4 , July 2010 . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-j18041XY.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Alexander Hornung, Aljoscha Smolic, and Markus Gross; Novel Stereoscopic Content Production Tools, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18041XY

Snippet:

Alexander Hornung, Aljoscha Smolic, and Markus Gross; Novel Stereoscopic Content Production Tools, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18041XY

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Alexander Hornung, Aljoscha Smolic, and Markus Gross; <cite>Novel Stereoscopic Content Production Tools</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at <a href="https://doi.org/10.5594/j18041XY" target="_blank" rel="noopener">https://doi.org/10.5594/j18041XY</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Alexander Hornung, Aljoscha Smolic, and Markus Gross; <cite id="bib-10-5594-j18041xy">Novel Stereoscopic Content Production Tools</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011
<span class="doi">10.5594/j18041XY</span>
</li>