Efficient Content-Driven Encoding Towards a Target Video Quality

Metadata

Publisher: SMPTE
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Motion Imaging Jour.
Volume: 134, No. 1, pp. 14–20
Abstract: Video streaming workflows aim to maximize video quality while still maintaining smooth video streaming performance. A traditional fixed bitrate ladder consists of predetermined bitrate-resolution pairs optimized across a wide variety of content. Consequently, these pairs are rarely optimized for a given piece of content. Some encoding tools address this by encoding each piece of video content with many codec parameters and then evaluating the results using a video quality metric. However, this process requires significant computation, which increases cost and encoding time. In this paper we propose a novel content-driven workflow that predicts optimal encoding parameters to achieve a target perceptual video quality. We do so by designing a deep learning model that, based on the video input, predicts a Video Multimethod Assessment Fusion (VMAF) rate-distortion curve. Our results indicate that such a content-driven approach efficiently reduces the number of encoding attempts, minimizes necessary cloud computing resources, encodes most efficiently, and maximizes perceptual video quality.
Publication Date: 2025-01-01
DOI: 10.5594/JMI.2025/JCRG7496
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2025/JCRG7496
Author(s): Trisha Mittal
Subhadra Gopalakrishnan
Jaclyn Pytlarz
Robin Atkins
Benjamin Rolling
Gabe Russell
Keyword(s): Bitrate Ladder, Streaming, Rate-Quality Curves, Video Coding, Video Compression
Copyright: © 2025 SMPTE

Bibliographic Reference(s)

Machine Learning for Per-Title Encoding [Active]
Rafael Huysegems “HTTP/2-Based Methods to Improve the Live Experience of Adaptive Streaming,” Proc. of the 23rd ACM International Conference on Multimedia, pp. 541–550. Oct. 2015. EXTERNAL
“How Netflix Pioneered Per-Title Video Encoding Optimization,” Streaming Media, Feb. 3, 2016. [Online]. Available: https://www.streamingmedia.com/Articles/Edito-rial/Featured-Articles/How-Netflix-Pioneered-Per-Title-Video-Encoding-Optimization-108547.aspx. [Accessed: Dec. 10, 2024]. EXTERNAL
Ahmed Telili, “Bitrate Ladder Prediction Methods for Adaptive Video Streaming: A Review and Benchmark.” arXiv preprint arXiv:2370.75763 (2023). EXTERNAL
Stefan Lederer, Christopher Müiler, and Christian Timmerer. “Dynamic adaptive streaming over HTTP dataset,” Proc. of the 3rd Multimedia Systems Conference, 2012, pp. 89–94, Feb. 2012. EXTERNAL
J. De Cock, Li, Z. M. Manohara, A. Aaron, “Complexity-based Consistent-quality Encoding in the Cloud,” Proc. 2076 IEEE International Conference on Image Processing (fCIP) 2076, pp. 1484–1488, Sep. 2016. EXTERNAL
C. Timmerer, H. Hellwagner, “Mipso: Multi-period perscene optimization for HTTP adaptive streaming,” Proc. 2020 IEEE International Conference on Multimedia and Expo (ICME) 2020, pp. 1–6 Jul. 2020. EXTERNAL
H. Amirpour, C. Timmerer, M. Ghanbari, “PSTR: Per-Title Encoding Using Spatio-Temporal Resolutions,” Proc. 2021 IEEE International Conference on Multimedia and Expo (ICME) 2021, pp. 1–6, Jul. 2021. EXTERNAL
“Per-Title Encode Optimization,” Netflix Tech Blog, Dec. 14, 2015. [Online]. Available: https://netflixtechblog.com/per-title-encode-optimization-7e99442b62a2. [Accessed: Dec. 10, 2024]. EXTERNAL
“VMAF: The Journey Continues,” Netflix Tech Blog, June 5, 2018. [Online]. Available: https://netflixtechblog.com/vmaf-the-journey-continues-44b51eeged12. [Accessed: Dec. 10, 2024]. EXTERNAL
Vilfredo Pareto, Manuel d'economie politique. Vol. 38. Giard & Briere, 1909. EXTERNAL
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600–612, Apr. 2004. EXTERNAL
“Dynamic Optimizer: A Perceptual Video Encoding Optimization Framework,” Netflix Tech Blog, Apr. 9, 2019. [Online]. Available: https://netflixtechblog.com/dynam-ic-optimizer-a-perceptual-video-encoding-optimization-framework-e19f1e3a277f. [Accessed: Dec. 10, Sep. 2024]. EXTERNAL
J. Suljug, S. Rimac-Drlje, “Fast Content Adaptive Representation Selection in HEVC-Based Video Coding for Streaming Applications,” Proc. Elmar -International Symposium Electronics in Marine. pp. 169–74, Sep. 2022. EXTERNAL
M. Bhat, J. M. Thiesse, P. Le Callet, “A case study of machine learning classifiers for real-time adaptive resolution prediction in video coding,” Proc. IEEE International Conference on Multimedia and Expo. pp. 1–6, Jul. 2020. EXTERNAL
V. V. Menon, H. Amirpour, M. Ghanbari, C. Timmerer, “Perceptually-Aware Per-Title Encoding for Adaptive Video Streaming,” Proc. IEEE International Conference on Multimedia and Expo, pp. 1–6, 2022. EXTERNAL
F. Nasiri, W. Hamidouche, L. Morin, N. Dholland, J.Y Aubie, “Ensemble Learning for Efficient VVC Bitrate Ladder Prediction,” Proc. the European Workshop on Visual Information Processing (EUVIP). 2022. [Online]. Available: http://arxiv.org/abs/2207.10317. EXTERNAL
A. V. Katsenou, J. Sole, D. Bull. “Efficient Bitrate Ladder Construction for Con-tent-Optimized Adaptive Video Streaming,” IEEE Open JSignal Process, 2: 496–511, Dec. 2021. EXTERNAL
A. Elahi, A. Falahati, F. Pakdaman, M. Modarressi, M. Gabbouj, “NCOD: Near-Opti-mum Video Compression for Object Detection,” Proc. IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5, May 2023. EXTERNAL
J. De Cock, Z. Li, M. Manohara, and A. Aaron, “Complexity-based consistent-quality encoding in the cloud,” Proc. 2016 IEEE International Conference on Image Processing (ICIP), pp. 1484–1488, Sep. 2016. EXTERNAL
M. Takeuchi, S. Saika, Y. Sakamoto, T. Nagashima, Z. Cheng, K. Kanai, J. Katto, K. Wei, J. Zengwei, and X. Wei, “Perceptual Quality Driven Adaptive Video Coding Using JND Estimation,” Proc. 2018 Picture Coding Symposium (PCS), pp. 179–183, May 2018. EXTERNAL
James V. Burke and Michael C. Ferris, “A Gauss-Newton Method for Convex Com-posite Optimization,” Mathematical Programming, 71(2): 179–194, Nov. 1995. EXTERNAL
C. Feichtenhofer, H. Fan, J. Malik, K. He, “SlowFast Networks for Video Recognition,” Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6202–6211, Oct. 2019. EXTERNAL
D. P. Kingma, J. Ba. “Adam: A Method for Stochastic Optimization.” [Online]. Available: arXiv preprint arXiv:1412.6980, Dec. 2014. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2025-JCRG7496.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; Efficient Content-Driven Encoding Towards a Target Video Quality, MIJ 2025, Volume 134, Number 1 (pp. 14 to 20); SMPTE, 2025. Available at https://doi.org/10.5594/JMI.2025/JCRG7496

Snippet:

Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; Efficient Content-Driven Encoding Towards a Target Video Quality, MIJ 2025, Volume 134, Number 1 (pp. 14 to 20); SMPTE, 2025. Available at https://doi.org/10.5594/JMI.2025/JCRG7496

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; <cite>Efficient Content-Driven Encoding Towards a Target Video Quality</cite>, MIJ 2025, Volume 134, Number 1 (pp. 14 to 20); SMPTE, 2025. Available at <a href="https://doi.org/10.5594/JMI.2025/JCRG7496" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2025/JCRG7496</a></span>

SMPTE's HTML Pub

Preview:

Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; Efficient Content-Driven Encoding Towards a Target Video Quality, MIJ 2025, Volume 134, Number 1 (pp. 14 to 20); SMPTE, 2025
doi: 10.5594/JMI.2025/JCRG7496
url: https://doi.org/10.5594/JMI.2025/JCRG7496

Snippet:

<li>
Trisha Mittal, Subhadra Gopalakrishnan, Jaclyn Pytlarz, Robin Atkins, Benjamin Rolling, and Gabe Russell; <cite id="bib-10-5594-jmi-2025-jcrg7496">Efficient Content-Driven Encoding Towards a Target Video Quality</cite>, MIJ 2025, Volume 134, Number 1 (pp. 14 to 20); SMPTE, 2025
<span class="doi">10.5594/JMI.2025/JCRG7496</span>
</li>