SMPTE Meetings and Conferences ( October 2019)
[ACTIVE]
Rotoscope Automation with Deep Learning
Metadata
- Publisher
- SMPTE — White Plains, NY, USA
- Doc Type
- Conference Paper
- Content Type
- Original Research
- Volume
- 2019, No. 10, pp. 1–9
- Abstract
- We present a deep learning based algorithm that can automatically rotoscope people on a given scene, without any user input. Current approaches to image matting require a significant amount of human input, whether by manually rotoscoping or by doing a chroma key. We show that this algorithm can perform comparably and even surpass the rotoscoping capabilities of After Effects' RotoBrush tool, in a variety of scenes comprising different lighting conditions, movements, and subjects. This makes it suitable for an integration within a VFX pipeline.
- Publication Date
- 2019-10-21
- DOI
10.5594/M001867- Link
- https://doi.org/10.5594/M001867
- Author(s)
- Oscar EstradaMotion Picture Science Rochester Institute of Technology 22 Green Isle Rd, Henrietta NY 14467Nicholas PerettiSoftware Engineering Rochester Institute of Technology Rochester, NYRicardo FigueroaFaculty Advisor Rochester Institute of Technology Rochester, NY
- Keyword(s)
- Instance Segmentation, Image Matting, Compositing, Deep Learning
- Copyright
- © 2019 Society of Motion Picture and Television Engineers, Inc.
Bibliographic Reference(s)
- T. Lin et al., "Microsoft COCO: Common Objects in Context", Computer Vision – ECCV 2014, pp. 740-755, 2014. Available: 10.1007/978-3-319-10602-1_48 [Accessed 2 June 2019]. EXTERNAL
- A. Karpathy, "CS231n Convolutional Neural Networks for Visual Recognition", Cs231n.github.io, 2019. [Online]. Available: http://cs231n.github.io. [Accessed: 02- Jun- 2019]. EXTERNAL
- M. Seymour, "The Art of Roto: 2011", fxguide, 2019. [Online]. Available: https://www.fxguide.com/featured/the-art-of-roto-2011/. [Accessed: 02-Jun- 2019]. EXTERNAL
- C. Singh, "Rotobrush", Cmsc426.github.io, 2019. [Online]. Available: https://cmsc426.github.io/rotobrush/. [Accessed: 02- Jun- 2019]. EXTERNAL
- C. Szegedy et al., "Going deeper with convolutions", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. Available: 10.1109/cvpr.2015.7298594 [Accessed 2 June 2019]. EXTERNAL
- S. Tsang, "Review: Xception — With Depthwise Separable Convolution, Better Than Inception-v3 (Image…", Towards Data Science, 2019. [Online]. Available: https://towardsdatascience.com/review-xception-with-depthwise-separable-convolution-better-than-inception-v3-image-dc967dd42568. [Accessed: 02- Jun- 2019]. EXTERNAL
- A. Smith and J. Blinn, "Blue screen matting", Proceedings of the 23rd annual conference on Computer graphics and interactive techniques -SIGGRAPH '96, 1996. Available: 10.1145/237170.237263 [Accessed 2 June 2019]. EXTERNAL
- F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. Available: 10.1109/cvpr.2017.195 [Accessed 2 June 2019]. EXTERNAL
- J. Olafenwa, "Review of Deep Residual Learning for Image Recognition", Medium, 2019. [Online]. Available: https://medium.com/deepreview/review-of-deep-residual-learning-for-image-recognition-a92955acf3aa. [Accessed: 02- Jun- 2019]. EXTERNAL
- D. Lowe, "Object recognition from local scale-invariant features", Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999. Available: 10.1109/iccv.1999.790410 [Accessed 2 June 2019]. EXTERNAL
- Y. Aksoy, T. Aydin, M. Pollefeys and A. Smolić, "Interactive High-Quality Green-Screen Keying via Color Unmixing", ACM Transactions on Graphics, vol. 36, no. 4, p. 1, 2016. Available: 10.1145/3072959.2907940 [Accessed 2 June 2019]. EXTERNAL
- Q. Zhu, P. Heng, L. Shao and X. Li, "What's the Role of Image Matting in Image Segmentation?", 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2013. Available: 10.1109/robio.2013.6739711 [Accessed 2 June 2019]. EXTERNAL
- X. Bai, J. Wang, D. Simons and G. Sapiro, "Video SnapCut", ACM SIGGRAPH 2009 papers on - SIGGRAPH '09, 2009. Available: 10.1145/1576246.1531376 [Accessed 2 June 2019]. EXTERNAL
- N. Xu, B. Price, S. Cohen and T. Huang, "Deep Image Matting", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. Available: 10.1109/cvpr.2017.41 [Accessed 2 June 2019]. EXTERNAL
- A. Rosebrock, "Intersection over Union (IoU) for object detection -PyImageSearch", PyImageSearch, 2019. [Online]. Available: https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/. [Accessed: 02- Jun- 2019]. EXTERNAL
- J. Hui, "Understanding Feature Pyramid Networks for object detection (FPN)", Medium, 2019. [Online]. Available: https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c. [Accessed: 02- Jun-2019]. EXTERNAL
- M. Everingham, L. Van Gool, C. Williams, J. Winn and A. Zisserman, "The Pascal Visual Object Classes (VOC) Challenge", International Journal of Computer Vision, vol. 88, no. 2, pp. 303-338, 2009. Available: 10.1007/s11263-009-0275-4 [Accessed 2 June 2019]. EXTERNAL
- J. Deng, W. Dong, R. Socher, L. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database", 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009. Available: 10.1109/cvpr.2009.5206848 [Accessed 2 June 2019]. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001867.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; Rotoscope Automation with Deep Learning, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019. Available at https://doi.org/10.5594/M001867
Snippet:
Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; Rotoscope Automation with Deep Learning, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019. Available at https://doi.org/10.5594/M001867
HTML (ISO 690 compliant)
Preview:
Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; Rotoscope Automation with Deep Learning, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019. Available at https://doi.org/10.5594/M001867
Snippet:
<span class="citation">Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; <cite>Rotoscope Automation with Deep Learning</cite>, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019. Available at <a href="https://doi.org/10.5594/M001867" target="_blank" rel="noopener">https://doi.org/10.5594/M001867</a></span>
SMPTE's HTML Pub
Preview:
Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; Rotoscope Automation with Deep Learning, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019
doi: 10.5594/M001867
url: https://doi.org/10.5594/M001867
doi: 10.5594/M001867
url: https://doi.org/10.5594/M001867
Snippet:
<li> Oscar Estrada, Nicholas Peretti, and Ricardo Figueroa; <cite id="bib-10-5594-m001867">Rotoscope Automation with Deep Learning</cite>, SMPTE Meetings and Conferences ( October 2019); SMPTE, 2019 <span class="doi">10.5594/M001867</span> </li>