Towards Efficient Multi-Codec Streaming

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2022, No. 10, pp. 1–16
Abstract: One of the biggest challenges in modern-era streaming is the fragmentation of codec support across receiving devices. For example, modern Apple devices can decode and seamlessly switch between H.264/AVC and HEVC streams. Most new TVs and set-top boxes can also decode HEVC, but they cannot switch between HEVC and H.264/AVC streams. And while most older devices/streaming clients can only receive and decode H.264/AVC streams. With the arrival of next-generation codecs - such as AV1 and VVC, the fragmentation of codec support across devices becomes even more complex. This situation brings a question - how can we serve such a population of devices most efficiently by using codecs delivering the best performance in all cases yet producing the minimum possible number of streams and such that the overall cost of media delivery is minimal? In this paper, we explain how this problem can be formalized and solved at the stage of dynamic generation of encoding profiles for ABR streaming. The proposed solution is a generalization of the context-aware encoding (CAE) class-of techniques, considering multiple sets of renditions generated using each codec and codec usage distributions by the population of the receiving devices. We also discuss several streaming system-level tools needed to make the proposed solution practically deployable.
Publication Date: 2022-10-24
DOI: 10.5594/M001963
ISBN: Print: 978-1-61482-963-8
Link: https://doi.org/10.5594/M001963
Author(s): Yuriy ReznikBrightcove Inc, Boston, MA, USA
Karl LillevoldBrightcove Inc, Boston, MA, USA
Abhijith JagannathBrightcove Inc, Boston, MA, USA
Nabajeet BarmanBrightcove UK Ltd, London, UK
Keyword(s): Adaptive Bitrate Streaming, DASH, HLS, H.264, HEVC, Multicodec Streaming, Per-Title Encoding, Context-aware Encoding, Content-Adaptive Encoding
Copyright: © 2022 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

Optimizing Mass-Scale Multi-Screen Video Delivery [Active]
Performance Comparison of Emerging EVC and VVC Video Coding Standards with HEVC and AV1 [Active]
CTA.5001-A — CTA-5001-A, “Web Application Video Ecosystem— Content Specification,” CTA, Dec. 2018. Accessed: Apr. 14, 2023. [Online]. Available: https://cdn.cta.tech/cta/media/media/resources/standards/pdfs/cta-5001-a-final.pdf EXTERNAL
ISO/IEC 14496-10 [Active]
ISO/IEC 23008-2 [Active]
ISO.23009-1.2019 — International Organization for Standardization/International Electrotechnical Commision (ISO/IEC) 23009-1:2019, “Information Technology—Dynamic Adaptive Streaming Over HTTP (DASH)—Part 1: Media Presentation Description and Segment Formats (4th Ed.),” 2019. [Online]. Available: https://www.iso.org/standard/79329.html EXTERNAL
ISO.23090-3.2021 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-3:2021, “Information Technology—Coded Representation of Immersive Media—Part 3: Versatile Video Coding,” ISO/IEC, Feb. 2021. EXTERNAL
"Can I use" web service, https://caniuse.com/, March 18, 2022. EXTERNAL
DASH-IF, "DASH-IF Interoperability Guidelines, v.3," https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf, DASH-IF, November 15, 2018. EXTERNAL
Y. Reznik, X. Li, K. O. Lillevold, A. Jagannath, and J. Greer, "Optimal Multi-Codec Adaptive Bitrate Streaming," in Proc. IEEE Int. Conf. Multimedia & Expo, Shanghai, China, 2019, pp. 348-353. EXTERNAL
Y. Reznik, "Average Performance of Adaptive Streaming," in Proc. Data Compression Conference (DCC'21), Snowbird, UT, 23-26 March 2021. EXTERNAL
Brightcove VideoCloud platform, https://www.brightcove.com/en/onlinevideo-platform EXTERNAL
G. J. Conklin, G. S. Greenbaum, K. O. Lillevold, A. F. Lippman, and Y. A. Reznik, "Video coding for streaming media delivery on the internet," IEEE Trans. CSVT, vol. 11, no. 3, pp. 269-281, 2001. EXTERNAL
P. Topiwala, M. Krishnan, and W. Dai, "Performance comparison of VVC, AV1, and HEVC on 8-bit and 10-bit content," Proc. SPIE 10752, Applications of Digital Image Processing XLI, September 17 2018. EXTERNAL
Brightcove Context-Aware Encoding, https://apis.support.brightcove.com/general/overview-context-aware-encoding.html EXTERNAL
D. Wu, Y.T. Hou, W. Zhu, Y-Q. Zhang, and JM Peha, "Streaming video over the internet: approaches and directions," IEEE Trans. CSVT, vol. 11, no. 3, pp. 282-300, 2001. EXTERNAL
T. Laude, Y. G. Adhisantoso, J. Voges, M. Munderloh and J. Ostermann, "A Comparison of JEM and AV1 with HEVC: Coding Tools, Coding Efficiency and Complexity," Picture Coding Symposium (PCS), 2018, pp. 36-40. EXTERNAL
B. Girod, M. Kalman, Y.J. Liang, and R. Zhang, "Advances in channel-adaptive video streaming," Wireless Comm. and Mobile Comp., vol. 2, no. 6, pp. 573-584, 2002. EXTERNAL
Y. Reznik, K. O. Lillevold, A. Jagannath, J. Greer, and J. Corley, "Optimal design of encoding profiles for ABR streaming," in Proc. Packet Video Workshop, Amsterdam, NL, June 12, 2018, pp. 43-47. EXTERNAL
Apple, "HTTP Live Streaming (HLS) Authoring Specification for Apple Devices," https://developer.apple.com/documentation/http_live_streaming/http_live_streaming_hls_authoring_specification_for_apple_devices, Apple, November 12, 2021. EXTERNAL
A. Aaron, Z. Li, M. Manohara, J. De Cock, and D. Ronca, "Per-title encode optimization," https://medium.com/netflixtechblog/per-title-encode-optimization-7e99442b62a2, Dec. 15 2015. EXTERNAL
AV1, "AV1 Bitstream and Decoding Process Specification, Version 1.0.0," Alliance for Open Media, January 18, 2019. EXTERNAL
C. Chen, Y. Lin, S. Benting, and A. Kokaram, "Optimized transcoding for large scale adaptive streaming using playback statistics," in Proc. IEEE Int. Conf. Image Proc., Oct 2018, pp. 3269-3273. EXTERNAL
Y. Reznik, J. Cenzano, B. Zhang, "Transitioning Broadcast to Cloud," Proc. 2020 NAB Broadcast Engineering and Information Technology Conference, Las Vegas, NV, May 13-14, 2020. EXTERNAL
R. Pantos and W. May, "HTTP live streaming, RFC 8216," IETF, 2017. EXTERNAL
RethinkTV research, "Media & Entertainment Transcoding Workload and Device Royalty Forecast 2020-2030", https://rethinkresearch.biz/reports-category/rethink-tv/#media-entertainment-transcoding-workload-and-device-royalty-forecast-2020-2030, April 30, 2021. EXTERNAL
R-REC-BT.500 — ITU-R Recommendation BT.500, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” International Telecommunications Union, Geneva, Switzerland, Jan. 2012. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001963.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Yuriy Reznik, Karl Lillevold, Abhijith Jagannath, and Nabajeet Barman; Towards Efficient Multi-Codec Streaming, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001963

Snippet:

Yuriy Reznik, Karl Lillevold, Abhijith Jagannath, and Nabajeet Barman; Towards Efficient Multi-Codec Streaming, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001963

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Yuriy Reznik, Karl Lillevold, Abhijith Jagannath, and Nabajeet Barman; <cite>Towards Efficient Multi-Codec Streaming</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at <a href="https://doi.org/10.5594/M001963" target="_blank" rel="noopener">https://doi.org/10.5594/M001963</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Yuriy Reznik, Karl Lillevold, Abhijith Jagannath, and Nabajeet Barman; <cite id="bib-10-5594-m001963">Towards Efficient Multi-Codec Streaming</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
<span class="doi">10.5594/M001963</span>
</li>