Unconstrained 2D to Stereoscopic 3D Image and Video Conversion using Semi-Automatic Energy Minimization Techniques

Metadata

Publisher: SMPTE — White Plains, NY
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2012, No. 10, pp. 1–12
Abstract: We present a method for semi-automatically converting unconstrained 2D images and video content into stereoscopic 3D. The user is presented with the image to convert, and brushes user-defined depth strokes in certain areas. These correspond to a rough estimate of the scene depths within these points. After, the rest of the depths are solved using this information, producing a depth map to create stereoscopic 3D content. For video, the user chooses several keyframes for brushing, and the depths for the entire video are found in a volumetric basis. Additionally for video, the user has the option of minimizing effort by employing a robust tracking algorithm, where the first frame only needs to be labeled. After, the labels are propagated throughout the entire video, ultimately increasing accuracy with more frames labeled. Our work combines the merits of two energy minimization techniques: Graph Cuts and Random Walks. The former respects boundaries, but does not have suitable depth diffusion, making the scene look like “cardboard cutouts”. The latter has good depth diffusion, but object boundaries are blurred. Therefore, combining the merits of both will lead to a higher quality result. Current efforts rely on automatic or manual conversion by rotoscopers. The former prohibits user intervention, while the latter is time consuming, prohibiting use in smaller studios. Semi-automatic is a compromise to allow for more faster and accurate conversion, decreasing the time for studios to release 3D content. The results shown in this paper generate good quality stereoscopic depth maps with minimal effort required.
Publication Date: 2012-10-01
DOI: 10.5594/M001453
Link: https://doi.org/10.5594/M001453
Author(s): Raymond PhanDepartment of Electrical & Computer Engineering, Ryerson University, 350 Victoria St., Toronto, ON, M5B 2K3, Canada
Richard RzeszutekDepartment of Electrical & Computer Engineering, Ryerson University, 350 Victoria St., Toronto, ON, M5B 2K3, Canada
Dimitrios AndroutsosDepartment of Electrical & Computer Engineering, Ryerson University, 350 Victoria St., Toronto, ON, M5B 2K3, Canada
Keyword(s): Stereoscopic
Copyright: © 2012 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

[1] Fehn C. de la Barre R. Pastoor S. , “Interactive 3-DTV: Concepts and Key Technologies” , Proc. of the IEEE , 94 ( 3 ): 524 – 538 , March 2006 . EXTERNAL
[2] Guttman M. Wolf L. Cohen-Or D. , “Semi-automatic Stereo Extraction from Video Footage” , Proc. IEEE ICCV , 2009 . EXTERNAL
[3] Boykov Y. Veksler O. Zabih R. , “Fast Approximate Energy Minimization via Graph Cuts” , IEEE Trans. on PAMI , 23 ( 11 ): 1222 – 1239 , 2002 . EXTERNAL
[4] Boykov Y. Funka-Lea G. , “Graph Cuts and Efficient N-D Image Segmentation” , Intl. Jnl. of Comp. Vis ., 2 ( 70 ): 109 – 131 , 2006 . EXTERNAL
[5] Grady L. , “Random Walks for Image Segmentation” , IEEE Trans. on PAMI , 28 ( 11 ): 1768 – 1783 , 2006 . EXTERNAL
[6] Rzeszutek R. El-Maraghi T. Androutsos D. , “Interactive Rotoscoping through Scale-Space Random Walks” , Proc. IEEE ICME , pp. 1334 – 1337 , 2009 . EXTERNAL
[7] Phan R. Rzeszutek R. Androutsos D. , “Semi-Automatic 2D to 3D Image Conversion using Scale-Space Random Walks and a Graph Cuts Based Depth Prior” , Proc. IEEE ICIP , pp. 865 – 868 , 2011 . EXTERNAL
[8] Kalal Z. Matas J. Mikolajczyk K. , “Online Learning of Robust Object Detectors during Unstable Tracking,” Proc. IEEE ICCV - 3rd On-Line Learning for Comp. Vis. Workshop , 2009 . EXTERNAL
[9] Kalal Z. Matas J. Mikolajczyk K. , “P-N Learning: Bootstrapping Binary Classifiers by Structural Constraints” , Proc. IEEE CVPR , 2010 . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001453.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Raymond Phan, Richard Rzeszutek, and Dimitrios Androutsos; Unconstrained 2D to Stereoscopic 3D Image and Video Conversion using Semi-Automatic Energy Minimization Techniques, SMPTE Meetings and Conferences ( October 2012); SMPTE, 2012. Available at https://doi.org/10.5594/M001453

Snippet:

Raymond Phan, Richard Rzeszutek, and Dimitrios Androutsos; Unconstrained 2D to Stereoscopic 3D Image and Video Conversion using Semi-Automatic Energy Minimization Techniques, SMPTE Meetings and Conferences ( October 2012); SMPTE, 2012. Available at https://doi.org/10.5594/M001453

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Raymond Phan, Richard Rzeszutek, and Dimitrios Androutsos; <cite>Unconstrained 2D to Stereoscopic 3D Image and Video Conversion using Semi-Automatic Energy Minimization Techniques</cite>, SMPTE Meetings and Conferences ( October 2012); SMPTE, 2012. Available at <a href="https://doi.org/10.5594/M001453" target="_blank" rel="noopener">https://doi.org/10.5594/M001453</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Raymond Phan, Richard Rzeszutek, and Dimitrios Androutsos; <cite id="bib-10-5594-m001453">Unconstrained 2D to Stereoscopic 3D Image and Video Conversion using Semi-Automatic Energy Minimization Techniques</cite>, SMPTE Meetings and Conferences ( October 2012); SMPTE, 2012
<span class="doi">10.5594/M001453</span>
</li>