Toward Efficient Multicodec Streaming
Metadata
- Publisher
- SMPTE — White Plains, NY, USA
- Doc Type
- Journal Article
- Content Type
- Original Research
- Abbreviated Title
- SMPTE Mot. Imag. J
- Volume
- 132, No. 4, pp. 16–25
- Abstract
- One of the biggest challenges in modern-era media delivery systems is the fragmentation of the population of receiving devices in terms of codec capabilities. For example, modern Apple devices can decode and seamlessly switch between H.264/Advanced Video Coding (AVC) and high efficiency video encoding (HEVC) streams. Most new TVs or set-top boxes can also decode HEVC, but they cannot switch between HEVC and H.264/AVC streams in the same streaming session. And there are still plenty of older devices/streaming clients that can only receive and decode H.264/AVC streams. With the arrival of next-generation codecs, such as AV1 and Versatile Video Coding (VVC), the fragmentation of codec support across devices becomes even more complex. This situation prompts the question—how can we serve such a population of devices most efficiently by using codecs delivering the best performance in all cases yet producing the minimum possible number of streams such that the overall cost of media delivery is minimal? In this article, we explain how this problem can be formalized and solved at the stage of dynamic generation of encoding profiles for adaptive bitrate (ABR) streaming. The proposed solution effectively generalizes the per-title or context-aware encoding (CAE) class of techniques, considering multiple sets of renditions generated using each codec and codec usage distributions by the population of the receiving devices. We also discuss additional system-level means (proper manifest generation, HLS and Dynamic Adaptive Streaming over HTTP (DASH) quality annotations, device detection, and edge logic) needed to make the proposed solution practically deployable.
- Publication Date
- 2023-05-01
- DOI
10.5594/JMI.2023.3263499- ISSN
- Print:
1545-0279| Electronic:2160-2492 - Link
- https://doi.org/10.5594/JMI.2023.3263499
- Author(s)
- Yuriy A. Reznik
bio
Yuriy A. Reznik is a technology fellow and the vice president of research at Brightcove Inc., Boston, MA. Previously, he held engineering and management positions with InterDigital, San Diego, CA, from 2011 to 2016; Qualcomm, San Diego, CA, from 2005 to 2011; and RealNetworks, Seattle, WA, from 1998 to 2005. In 2008, he was a visiting scholar at Stanford University, Stanford, CA. Since 2001, he has been involved in the work of ITU-T SG16 and MPEG standards committees and made contributions to several multimedia coding and delivery standards, including ITU-T H.264/MPEG-4 Advanced Video Coding (AVC), MPEG-4 ALS, ITU-T G.718, ITU-T H.265/MPEG-HEVC, and MPEG-Dynamic Adaptive Streaming over HTTP (DASH). Several technologies, standards, and products that he has helped to develop (RealAudio/RealVideo, ITU-T H.264/MPEG-4 AVC, Zencoder, Brightcove CAE, and MPEG-DASH) have been recognized by the NATAS Technology and Engineering Emmy Awards. He received a PhD degree in computer science from Kyiv University, Kyiv, Ukraine. He is a Senior Member of IEEE and the SPIE, and a Member of the ACM, AES, and SMPTE. He is a coauthor of over 150 conference papers and journal articles, and a coinventor of over 70 granted U.S. patents. - Keyword(s)
- Adaptive Bitrate Streaming, Multicodec Streaming, HLS, DASH, H.264, HEVC, AV1
- Copyright
- © 2023 SMPTE
Bibliographic Reference(s)
- Optimizing Mass-Scale Multi-Screen Video Delivery [Active]
- Performance Comparison of Emerging EVC and VVC Video Coding Standards with HEVC and AV1 [Active]
- CTA.5001-A — CTA-5001-A, “Web Application Video Ecosystem— Content Specification,” CTA, Dec. 2018. Accessed: Apr. 14, 2023. [Online]. Available: https://cdn.cta.tech/cta/media/media/resources/standards/pdfs/cta-5001-a-final.pdf EXTERNAL
- ISO/IEC 14496-10 [Active]
- ISO/IEC 23008-2 [Active]
- ISO.23009-1.2019 — International Organization for Standardization/International Electrotechnical Commision (ISO/IEC) 23009-1:2019, “Information Technology—Dynamic Adaptive Streaming Over HTTP (DASH)—Part 1: Media Presentation Description and Segment Formats (4th Ed.),” 2019. [Online]. Available: https://www.iso.org/standard/79329.html EXTERNAL
- ISO.23090-3.2021 — International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 23090-3:2021, “Information Technology—Coded Representation of Immersive Media—Part 3: Versatile Video Coding,” ISO/IEC, Feb. 2021. EXTERNAL
- Brightcove Context-Aware Encoding. Accessed: Apr. 14, 2023. [Online]. Available: https://apis.support.brightcove.com/general/overview-context-aware-encoding.html EXTERNAL
- B. Girod, M. Kalman, Y. J. Liang, and R. Zhang, “Advances in Channel-Adaptive Video Streaming,” Wireless Commun. Mobile Comput., 2(6):573–584, 2002. EXTERNAL
- RethinkTV Research, “Media & Entertainment Transcoding Workload and Device Royalty Forecast 2020-2030,” Apr. 30, 2021. Accessed: Apr. 14, 2023. [Online]. Available: https://rethinkresearch.biz/reports-category/rethink-tv/#media-entertainment-transcod-ing-workload-and-device-royalty-forecast-2020-2030 EXTERNAL
- “Can I Use” Web Service. Mar. 18, 2022. Accessed: Apr. 14, 2023. [Online]. Available: https://caniuse.com/ EXTERNAL
- Y. Reznik, J. Cenzano, and B. Zhang, “Transitioning Broadcast to Cloud,” Proc. 2020 NAB Broadcast Eng. and Inf. Technol. Conf., Las Vegas, NV, May 13–14, 2020. EXTERNAL
- A. Aaron, Z. Li, M. Manohara, J. De Cock, and D. Ronca, “Per-Title Encode Optimization,” Dec. 15, 2015. Accessed: Apr. 14, 2023. [Online]. Available: https://medium.com/netflixtechblog/per-title-encode-optimization-7e99442b62a2 EXTERNAL
- G. J. Conklin, G. S. Greenbaum, K. O. Lillevold, A. F. Lippman, and Y. A. Reznik, “Video Coding for Streaming Media Delivery on the Internet,” IEEE Trans. CSVT, 11(3):269–281, 2001. EXTERNAL
- Y. Reznik, K. O. Lillevold, A. Jagannath, J. Greer, and J. Corley, “Optimal Design of Encoding Profiles for ABR Streaming,” Proc. Packet Video Workshop, Amsterdam, NL, pp. 43–47, Jun. 12, 2018. EXTERNAL
- AV1, “AV1 Bitstream and Decoding Process Specification, Version 1.0.0,” Alliance for Open Media, Jan. 18, 2019. EXTERNAL
- D. Wu, Y. T. Hou, W. Zhu, Y.-Q. Zhang, and J. M. Peha, “Streaming Video Over the Internet: Approaches and Directions,” IEEE Trans. CSVT, 11(3):282–300, 2001. EXTERNAL
- T. Laude, Y. G. Adhisantoso, J. Voges, M. Munderloh, and J. Ostermann, “A Comparison of JEM and AV1 with HEVC: Coding Tools, Coding Efficiency and Complexity,” 2018 Picture Coding Symp. (PCS), pp. 36–40, 2018. EXTERNAL
- Y. Reznik, “Average Performance of Adaptive Streaming,” Proc. Data Compression Conf. (DCC’21), Snowbird, UT, Mar. 23–26, 2021. EXTERNAL
- R. Pantos and W. May, “HTTP Live Streaming, RFC 8216,” Internet Engneering Taskforce (IETF), 2017. EXTERNAL
- Brightcove VideoCloud Platform. Accessed: Apr. 14, 2023. [Online]. Available: https://www.brightcove.com/en/products/online-video-platform/ EXTERNAL
- C. Chen, Y. Lin, S. Benting, and A. Kokaram, “Optimized Transcoding for Large Scale Adaptive Streaming Using Playback Statistics,” 2018 25th Proc. IEEE Int. Conf. on Image Process., pp. 3269–3273, Oct. 2018. EXTERNAL
- Y. Reznik, X. Li, K. O. Lillevold, A. Jagannath, and J. Greer, “Optimal Multi-Codec Adaptive Bitrate Streaming,” Proc. IEEE Int. Conf. Multimedia & Expo, Shanghai, China, pp. 348–353, 2019. EXTERNAL
- P. Topiwala, M. Krishnan, and W. Dai, “Performance Comparison of VVC, AV1, and HEVC on 8-Bit and 10-Bit Content,” Proc. SPIE 10752, Appl. of Digital Image Process. XLI, Sep. 17, 2018. EXTERNAL
- DASH-IF, “DASH-IF Interoperability Guidelines, v.3,” DASH-IF, Nov. 15, 2018. Accessed: Apr. 14, 2023. [Online]. Available: https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf EXTERNAL
- Apple, “HTTP Live Streaming (HLS) Authoring Specification for Apple Devices,” Apple, Nov. 12, 2021. Accessed: Apr. 14, 2023. [Online]. Available: https://developer.apple.com/documentation/http_live_streaming/http_live_streaming_hls_authoring_specification_for_apple_devices EXTERNAL
- International Telecommunication Union—Radiocommunication (ITU-R) Rec. BT.500, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” ITU-R, Oct. 2019. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2023.3263499.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Yuriy A. Reznik; Toward Efficient Multicodec Streaming, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3263499
Snippet:
Yuriy A. Reznik; Toward Efficient Multicodec Streaming, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3263499
HTML (ISO 690 compliant)
Preview:
Yuriy A. Reznik; Toward Efficient Multicodec Streaming, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3263499
Snippet:
<span class="citation">Yuriy A. Reznik; <cite>Toward Efficient Multicodec Streaming</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023. Available at <a href="https://doi.org/10.5594/JMI.2023.3263499" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2023.3263499</a></span>
SMPTE's HTML Pub
Preview:
Yuriy A. Reznik; Toward Efficient Multicodec Streaming, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023
doi: 10.5594/JMI.2023.3263499
url: https://doi.org/10.5594/JMI.2023.3263499
doi: 10.5594/JMI.2023.3263499
url: https://doi.org/10.5594/JMI.2023.3263499
Snippet:
<li> Yuriy A. Reznik; <cite id="bib-10-5594-jmi-2023-3263499">Toward Efficient Multicodec Streaming</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 4, May 2023); SMPTE, 2023 <span class="doi">10.5594/JMI.2023.3263499</span> </li>