Computational Imaging for Stop-Motion Animated Video Productions

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 125, No. 1, pp. 42–47
Abstract: Creating movies using stop-motion technology remains a fascinating approach for storytelling even in the era of digital cinema. A huge number of professional and semiprofessional clips that were produced as fan art can be watched on Internet video platforms and serve as testimonial for the unbroken interest in unleashing creative potential using this technology. However, producing content using stop motion remains a cumbersome task that raises production costs, especially for full-length movies. Consequently, a trend can be observed even for successful television series that were originally produced using stop motion in which the production scheme was changed to computer animation. Against this background, we propose a production scheme for stop-motion-animated movies that has the potential to lower the production costs while increasing the quality of the resulting content. By using a static multicamera array and algorithms from the field of computational imaging, our technology allows artistic effects to be created in the post-production that are difficult to realize using conventional stop-motion production methods. Our approach allows changing the depth of field, smoothly moving the camera along virtual camera paths, and upsampling the frame rate of the stop-motion video in high quality. All effects are computed and applied in post-production, while all intrinsic and extrinsic parameters of the cameras remain static during the whole production. To demonstrate the practicability, results are shown from a stop-motion video that has been produced using the proposed approach.
Publication Date: 2016-01-01
DOI: 10.5594/j18663
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/j18663
Author(s): Frederik Zilly
bio
Frederik Zilly is head of the Computational Imaging and Algorithms group in the Moving Picture Department of Fraunhofer IIS in Erlangen, Germany. He received a diploma degree in physics from Free University of Berlin, and a Ph.D. degree in computer science from Technical University, Berlin. Before joining Fraunhofer IIS, he was scientific project manager in the Immersive Media and 3D Video group in the Image Processing Department at Fraunhofer HHI in Berlin where he was involved in several German and European research projects related to 3DTV. In this function, Zilly was mainly responsible for the development of the stereoscopic analyzer (STAN) and coordinated the activities of the Fraunhofer HHI concerning the European research project MUSCADE. Zilly has been honored for his work on the assistance system STAN (Stereoscopic Analyzer) with the Award for Outstanding Merit in Young Scientists, the Rudolf Urtel Prize 2011. He serves as reviewer for different international conferences and journals. His research focus lies in multicamera image processing algorithms.
Matthias Ziegler
bio
Matthias Ziegler received a diploma in electrical engineering from Friedrich-Alexander University Erlangen, Germany, 2012. He has been with Fraunhofer IIS since that time and is working on algorithms for computational imaging. His research focus is on light-field production workflows and algorithms, including disparity estimation, plenoptic cameras, and high-dynamic range.
Joachim Keinert
bio
Joachim Keinert, chief scientist at Fraunhofer IIS, has diplomas from both the University of Stuttgart, Germany, and Télécom ParisTech (formerly ENST) in Paris, France. In 2009, he received a Ph.D. with honors in the domain of electronic system level design for image processing applications. His research interests focus on high-dynamic range video capture and processing as well as light-field imaging and depth acquisition. Keinert actively participates in the JPEG compression standardization where he co-chairs the Systems group. He is experienced in development of hard- and software for image processing in both embedded and general-purpose systems. Currently, he leads several image processing projects in the context of light-field imaging and 3D capturing.
Michael Schoberl
bio
Michael Schöberl received a diploma in electrical engineering with honors in 2006 from the University of Erlangen-Nuremberg. He then joined the Fraunhofer IIS and started to work on high-quality imaging systems in the Electronic Imaging department. In 2007, he joined the Chair of Multimedia Communications and Signal Processing at the University of Erlangen-Nuremberg to continue the research on theory and algorithms for high-end camera systems in a close cooperation with Fraunhofer IIS. In 2013, he received a Ph.D. with honors in electrical engineering for his work on digital cameras. He is currently working as a chief scientist in the Electronic Imaging department at Fraunhofer IIS, with a focus on research and innovation management.
Siegfried Foessel
bio
Siegfried Foessel is head of the Moving Picture Technologies department at the Fraunhofer IIS in Erlangen and spokesman of the Fraunhofer digital cinema alliance. He was responsible for projects such as the camera development ARRI D20/D21, the DCI (Digital Cinema Initiatives) certification test plan, and the JPEG 2000 standardization for digital cinema. Foessel is a member of various standardization bodies and organizations such as SMPTE, Fernseh-und Kinotechische Gesellschaft e.V. (FKTG), the International Organization for Standardization (ISO), and the German Institute for Standardization (Deutsches Institut für Normung e.V [DIN]). He became a SMPTE Fellow in 2014. Foessel was the 2014 recipient of the Josef-von-Fraunhofer award for the development of easyDCP, a tool for creating and playback of Digital Cinema Packages. In 2014, he was elected president of the FKTG, the German equivalent to SMPTE.
Copyright: © 2016 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

Light-Field Acquisition and Processing System for Film Productions [Active]
C. Fehn, “A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR),” Proc. Visualization, Imaging, and Image Processing (VIIP 2003), pp. 482-497, 2003. EXTERNAL
R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, 2005, “Light Field Photography with a Hand-held Plenoptic Camera,” Computer Science Technical Report, Stanford, CA, https://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf. EXTERNAL
M. Ziegler, Engelhardt A., Müller S., Keinert J., Foessel S., Schmid K., “Multi-camera system for depth based visual effects and compositing”, Proc. 12th European Conference on Visual Media Production (CVMP ’15), 2015. EXTERNAL
M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIG-GRAPH, 1996, pp. 31-42, 1996. EXTERNAL
Riechert C., Zilly F., Müller M., Kauff P., „Advanced Interpolation for Depth Image Based Rendering“, 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), EXTERNAL
F. Zilly, M. Schöberl, P. Schäfer, M. Ziegler, J. Keinert, and S. Foessel, “Lightfield Media Production System Using Sparse Angular Sampling,” presented at the ACM SIGGRAPH, Anaheim, CA, July 2013. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-j18663.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Frederik Zilly, Matthias Ziegler, Joachim Keinert, Michael Schoberl, and Siegfried Foessel; Computational Imaging for Stop-Motion Animated Video Productions, SMPTE Motion Imaging Journal ( Volume: 125, Issue: 1, 2016); SMPTE, 2016. Available at https://doi.org/10.5594/j18663

Snippet:

Frederik Zilly, Matthias Ziegler, Joachim Keinert, Michael Schoberl, and Siegfried Foessel; Computational Imaging for Stop-Motion Animated Video Productions, SMPTE Motion Imaging Journal ( Volume: 125, Issue: 1, 2016); SMPTE, 2016. Available at https://doi.org/10.5594/j18663

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Frederik Zilly, Matthias Ziegler, Joachim Keinert, Michael Schoberl, and Siegfried Foessel; <cite>Computational Imaging for Stop-Motion Animated Video Productions</cite>, SMPTE Motion Imaging Journal ( Volume: 125, Issue: 1, 2016); SMPTE, 2016. Available at <a href="https://doi.org/10.5594/j18663" target="_blank" rel="noopener">https://doi.org/10.5594/j18663</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Frederik Zilly, Matthias Ziegler, Joachim Keinert, Michael Schoberl, and Siegfried Foessel; <cite id="bib-10-5594-j18663">Computational Imaging for Stop-Motion Animated Video Productions</cite>, SMPTE Motion Imaging Journal ( Volume: 125, Issue: 1, 2016); SMPTE, 2016
<span class="doi">10.5594/j18663</span>
</li>