A Nonconventional Approach to the Conversion of 2D Video and Film Content to Stereoscopic 3D

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 120, No. 4, pp. 41–48
Abstract: This paper presents a nonconventional approach and method for converting naturalistic (non–computer-generated) two-dimensional (2D) video and film material to Stereoscopic 3D. Experimental evidence is presented to show the efficacy of color-based surrogate depth maps for automatic 2D-to-3D conversion aimed at small-screen applications. A semiautomatic 2D-to-3D conversion system (CRC-DMEG), also based on surrogate depth maps, for the conversion of video and film content for commercial projection on large cinema screens is then presented. A major advantage of CRC-DMEG is that it exploits the correlation between the 2D color images and the surrogate depth maps to allow for direct manipulation of the depth of objects in a scene. Another major advantage is that it allows for instant realization of depth details, such as raindrops, foliage, and textures found in carpets. This approach and method minimizes the labor-intensive work associated with the conventional method of rotoscoping. Moreover, the manual task of filling in disoccluded regions is also significantly reduced.
Publication Date: 2011-05-01
DOI: 10.5594/j18043XY
ISSN: Print: 1545-0279
Link: https://doi.org/10.5594/j18043XY
Author(s): Carlos Vazquez
bio
Carlos Vázquez, research scientist, received a B. Eng. and M.Sc. degrees from the Higher Polytechnic Institute José Antonio Echeverría (ISPJAE) in Havana, Cuba, where he also worked as a lecturer and assistant professor. He received a Ph.D. degree in telecommunications from the INRS-EMT, Montréal, Canada, in 2003, following his position as a Research Associate at Concordia University, Montréal, Canada, until 2005. In August 2005 he joined the Communications Research Centre (CRC), which is the Canadian government's primary laboratory for R&D in advanced telecommunications. His research interests are in the areas of 3DTV; stereo and multiview vision systems; image/video representation, sampling and interpolation; image and video coding; and motion/disparity/depth estimation and compensation.
Wa James Tam
bio
Wa James Tam is a senior research scientist at the CRC. Tam holds a Ph.D. in experimental psychology from York University, Toronto, and has been with the CRC for more than 20 years. Tam's research interest is in human visual perception and its application to the design and development of advanced video systems. One of the first projects that he worked on was eye movements and television viewing. In the early 90s, he co-founded the 3D research program at CRC. He is currently a project leader on 3DTV in the Advanced Video Systems Group in the Broadcast Technologies Branch.
Copyright: © 2011 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

1 Yamada K. Suehiro K. Nakamura H. , “Pseudo 3D Image Generation with Simple Depth Models,” International Conference on Consumer Electronics, Digest of Technical Papers , pp. 277 – 278 , Jan. 2005 . EXTERNAL
2 Vázquez C. Tam W. J. , “CRC-CSDM: 2D-to-3D Conversion Using Colour-Based Surrogate Depth Maps,” Proc. IEEE 3D Systems and Applications Workshop 2010 , pp. 1194 – 1205 , May 2010 . EXTERNAL
3 Battiato S. Curti S. La Cascia M. Tortora M. Scordato E. , “Depth Map Generation by Image Classification,” Proc. SPIE Conference in Three-Dimensional Image Capture and Applications VI , vol. 5302 , pp. 95 – 104 , April 2004 . EXTERNAL
4 Geshwind David M. Handal Anthony H. U.S. Patent No. 4 925 294, May 15, 1990. EXTERNAL
5 Kaye M. C. Best C. J. L. , U.S. Patent No. 7 102 633, Sep. 5, 2006. EXTERNAL
6 Tauber Z. Ze-Nian L. Drew M. S. , “Review and Preview: Disocclusion by Inpainting for Image-Based Rendering,” IEEE Trans. Syst. Man Cybern. C Appl. Rev ., 37 ( 4 ): 527 – 540 , July 2007 . EXTERNAL
7 Harman P. Flack J. Fox S. Dowley M. , “Rapid 2D to 3D Conversion,” SPIE Conference on Stereoscopic Displays and Virtual Reality Systems IX , 4660 : 78 – 86 , 2002 . EXTERNAL
8 Tam W. J. Speranza F. Zhang L. , “Depth Map Generation for 3D-TV: Importance of Edge and Boundary Information,” in Advances in Three-Dimensional Television, Video and Display , pp. 153 – 181 , Springer : New York , 2009 . EXTERNAL
9 Tam W. J. Vázquez C. Speranza F. , “Surrogate Depth Maps for Stereoscopic Imaging: Diff erent Edge Types,” Proc. SPIE Conference on Stereoscopic Displays and Virtual Reality Systems XIV , 6490 : 0C1 – 0C9 , Jan. 2007 . EXTERNAL
10 Tam W. J. Vázquez C. Speranza F. , “Three-Dimensional TV: A Novel Method for Generating Surrogate Depth Maps Using Colour Information,” Proc. SPIE Conference on Stereoscopic Displays and Applications XX , pp. 7237 – 7248 , Jan. 2009 . EXTERNAL
11 Tam W. J. Vázquez C. , “Method and Graphical User Interface for Modifying Depth Maps,” U.S. Patent Application No. 12 508 208, Jul. 23, 2009. EXTERNAL
12 Tam W. J. Vázquez C. , “Generation of a Depth Map from a Monoscopic Color Image for Rendering Stereoscopic Still and Video Images,” U.S. Patent Application No. 12 060 978, Apr. 2, 2008. EXTERNAL
13 Fehn C. , “Depth-Image–Based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV,” Proc. SPIE Conference on Stereoscopic Displays and Virtual Reality Systems XI , 5291 : 93 – 104 , Jan. 2004 . EXTERNAL
14 Tam W. J. Stelmach L. B. Meegan D. Vincent A. , “Bandwidth Reduction for Stereoscopic Video Signals,” Proc. SPIE Conference on Stereoscopic Displays and Applications VII , 3957 : 33 – 40 , Jan. 2000 . EXTERNAL
15 Stelmach L. B. Tam W. J. Meegan D. Vincent A. , “Stereo Image Coding Based on Human Visual Characteristics: Mixed Resolution and Unequal Bit Allocation,” International Workshop on Synthetic-Natural Hybrid Coding and Three-Dimensional Imaging (IWSNHC3DI'99 ), Santorini, Greece , Sep. 1999 . EXTERNAL
16 Stelmach L. B. Tam W. J. Meegan D. , “Stereo Image Quality: Eff ects of Spatiotemporal Resolution,” Proc. Stereoscopic Displays and Virtual Reality Systems VI , 3639 : 4 – 11 , Jan. 1999 . EXTERNAL
17 ISO/IEC JTC 1/SC 29/WG11, Doc. ISO/IEC FDIS 23002-3:2007(E) , “Part 3: Representation of Auxiliary Video and Supplemental Information,” Jan. 2007 . EXTERNAL
18 Zhang L. Tam W. J. , “Stereoscopic Image Generation Based on Depth Images for 3D TV,” IEEE Transactions on Broadcasting , 51 ( 2 ): 191 – 199 , 2005 . EXTERNAL
19 Vázquez C. Tam W. J. Speranza F. , “Stereoscopic Imaging: Filling Disoccluded Areas in Image-Based Rendering,” Proc. SPIE Conference on 3D TV, Video, and Display (ITCOM ), 6392 : 0D1 – 0D12 , Oct. 2006 . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-j18043XY.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Carlos Vazquez and Wa James Tam; A Nonconventional Approach to the Conversion of 2D Video and Film Content to Stereoscopic 3D, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18043XY

Snippet:

Carlos Vazquez and Wa James Tam; A Nonconventional Approach to the Conversion of 2D Video and Film Content to Stereoscopic 3D, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18043XY

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Carlos Vazquez and Wa James Tam; <cite>A Nonconventional Approach to the Conversion of 2D Video and Film Content to Stereoscopic 3D</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011. Available at <a href="https://doi.org/10.5594/j18043XY" target="_blank" rel="noopener">https://doi.org/10.5594/j18043XY</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Carlos Vazquez and Wa James Tam; <cite id="bib-10-5594-j18043xy">A Nonconventional Approach to the Conversion of 2D Video and Film Content to Stereoscopic 3D</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 4, May 2011); SMPTE, 2011
<span class="doi">10.5594/j18043XY</span>
</li>