Efficient Content-Driven Encoding Towards a Target Video Quality

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–11
Abstract: Video streaming workflows aim to maximize video quality while still maintaining smooth video streaming performance. A traditional fixed bitrate ladder consists of predetermined bitrate-resolution pairs which are optimized across a wide variety of content. Consequently, these pairs are rarely optimized for a given piece of content. Some encoding tools address this by encoding each piece of video content with many codec parameters and then evaluating the results using a video quality metric. However, this process requires significant computation which increases cost and encoding time. In this paper, we propose a novel content-driven workflow that predicts optimal encoding parameters to achieve a target perceptual video quality. We do so by designing a deep learning model that, based on the video input, predicts a VMAF rate-distortion curve. Our results indicate that such a content-driven approach is an efficient way to reduce the number of encoding attempts, minimize necessary cloud computing resources, encode most efficiently, and maximize perceptual video quality.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3048
ISBN: Electronic: 978-1-61482-965-2
Link: https://doi.org/10.5594/MOO/3048
Author(s): Trisha Mittal
Subhadra Gopalakrishnan
Jaclyn Pytlarz
Robin Atkins
Benjamin Rolling
Gabe Russell
Keyword(s): Bitrate Ladder, Streaming, Rate-Quality Curves, Video Coding, Video Compression, Video Quality, VMAF
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

Machine Learning for Per-Title Encoding [Active]
Huysegems, Rafael, “HTTP/2-based methods to improve the live experience of adaptive streaming.” Proceedings of the 23rd ACM international conference on Multimedia. 2015. EXTERNAL
How Netflix Pioneered Per-Title Video Encoding Optimization [Internet]. Streaming Media Magazine. Available from: https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=143238 EXTERNAL
Bitrate Ladder Prediction Methods for Adaptive Video Streaming: A Review and Benchmark [Internet]. ar5iv.org. Available from: https://ar5iv.labs.arxiv.org/html/2310.15163 EXTERNAL
Lederer S, Müller C, Timmerer C. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd multimedia systems conference 2012 Feb 22 (pp. 89–94). EXTERNAL
De Cock J, Li Z, Manohara M, Aaron A. Complexity-based consistent-quality encoding in the cloud. In 2016 IEEE International Conference on Image Processing (ICIP) 2016 Sep 25 (pp. 1484–1488). IEEE. EXTERNAL
Timmerer C, Hellwagner H. Mipso: Multi-period per-scene optimization for http adaptive streaming. In 2020 IEEE International Conference on Multimedia and Expo (ICME) 2020 Jul 6 (pp. 1–6). IEEE. EXTERNAL
Amirpour H, Timmerer C, Ghanbari M. PSTR: Per-Title Encoding Using Spatio-Temporal Resolutions. In 2021 IEEE International Conference on Multimedia and Expo (ICME) 2021 Jul 5 (pp. 1–6). IEEE. EXTERNAL
Netflix Technology Blog. Per-Title Encode Optimization [Internet]. 2015 Dec 14 [cited 2024 Jul 15]. Available from: https://netflixtechblog.com/per-title-encode-optimization-7e99442b62a2 EXTERNAL
Netflix. VMAF: The Journey Continues. Netflix Tech Blog [Internet]. 2018 Jun 28 [cited 2024 Jul 15]. Available from: https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12. EXTERNAL
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. EXTERNAL
Netflix. Dynamic optimizer: A perceptual video encoding optimization framework. Netflix Tech Blog [Internet]. 2019 Sep 5 [cited 2024 Jul 15]. Available from: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-video-encoding-optimization-framework-e19fle3a277f. EXTERNAL
Suljug J, Rimac-Drlje S. Fast Content Adaptive Representation Selection in HEVC- Based Video Coding for Streaming Applications. In: Proceedings Elmar - International Symposium Electronics in Marine. 2022. p. 169–74. EXTERNAL
Bhat M, Thiesse JM, Le Callet P. A case study of machine learning classifiers for real-time adaptive resolution prediction in video coding. In: Proceedings - IEEE International Conference on Multimedia and Expo. 2020. p. 1–6. EXTERNAL
Menon VV, Amirpour H, Ghanbari M, Timmerer C. Perceptually-Aware Per-Title Encoding for Adaptive Video Streaming. In: Proceedings - IEEE International Conference on Multimedia and Expo. 2022. p. 1–6. EXTERNAL
Nasiri F, Hamidouche W, Morin L, Dholland N, Aubie JY. Ensemble Learning for Efficient VVC Bitrate Ladder Prediction. In: Proceedings of the European Workshop on Visual Information Processing (EUVIP). 2022. Available from: http://arxiv.org/abs/2207.10317. EXTERNAL
Katsenou AV, Sole J, Bull D. Efficient Bitrate Ladder Construction for Content-Optimized Adaptive Video Streaming. IEEE Open J Signal Process. 2021; 2: 496–511. EXTERNAL
Elahi A, Falahati A, Pakdaman F, Modarressi M, Gabbouj M. NCOD: Near-Optimum Video Compression for Object Detection. In: IEEE International Symposium on Circuits and Systems (ISCAS). 2023. p. 1–5. EXTERNAL
J. De Cock, Z. Li, M. Manohara, and A. Aaron, “Complexity-based consistent-quality encoding in the cloud,” in 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 1484–1488. EXTERNAL
M. Takeuchi, S. Saika, Y. Sakamoto, T. Nagashima, Z. Cheng, K. Kanai, J. Katto, K. Wei, J. Zengwei, and X. Wei, “Perceptual quality driven adaptive video coding using jnd estimation,” in 2018 Picture Coding Symposium (PCS), 2018, pp. 179–183. EXTERNAL
Burke, James V., and Michael C. Ferris. “A Gauss-Newton method for convex composite optimization.” Mathematical Programming 71.2 (1995): 179–194. EXTERNAL
Feichtenhofer C, Fan H, Malik J, He K. SlowFast Networks for Video Recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2019. p. 6202–6211. EXTERNAL
Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980. 2014. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3048.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; Efficient Content-Driven Encoding Towards a Target Video Quality, MTS 2024, Article 37 (pp. 1 to 11); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3048

Snippet:

Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; Efficient Content-Driven Encoding Towards a Target Video Quality, MTS 2024, Article 37 (pp. 1 to 11); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3048

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; <cite>Efficient Content-Driven Encoding Towards a Target Video Quality</cite>, MTS 2024, Article 37 (pp. 1 to 11); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at <a href="https://doi.org/10.5594/MOO/3048" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3048</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; <cite id="bib-10-5594-moo-3048">Efficient Content-Driven Encoding Towards a Target Video Quality</cite>, MTS 2024, Article 37 (pp. 1 to 11); SMPTE, 2024, ISBN: 978-1-61482-965-2
<span class="doi">10.5594/MOO/3048</span>
</li>