Mastering Quantization is Key for Video Compression

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 131, No. 5, pp. 45–53
Abstract: This article discusses some key elements and solutions for efficient video compression. More precisely, we introduce key differentiating techniques for video signal quantization within modern hybrid video codecs. First, we recall the fundamentals of scalar quantization and rate-distortion (R-D) theory and then describe the different control points and levels of granularity for optimizing signal quantization in modern video compression standards. From this common knowledge, we discuss approaches in how to jointly optimize those various levels of quantization refinements for improved video encoding quality. Most notably, we explain how to take advantage of a look-ahead module (available in most industrial encoder implementations) to model spatio-temporal coding dependencies, and how to further compute optimal quantization information from an R-D standpoint for a group of pictures (GOPs). Complementary to this first approach, we share some insights about a local quantization refinement (LQR) algorithm. Such an algorithm is often ignored in practical encoder implementation due to its apparent complexity; we develop how and why it can efficiently work in realtime software encoding.
Publication Date: 2022-06-01
DOI: 10.5594/JMI.2022.3167782
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2022.3167782
Author(s): Michaël Ropert
bio
Michaël Ropert is currently responsible for innovation activities for video delivery solutions at MediaKind,Cesson-Sévigné, France. He led the Advanced Video Team, Ericsson (formerly Envivio), France, until 2019. His research interests are signal and image processing. He earned a PhD degree from the University of Rennes 1, Rennes, France, in 1995. From 1991 to 1992, he was a teacher in applied mathematics. While at Orange Labs, Cesson-Sévigné, he worked in the areas of the image and video compression for multimedia and digital television, from 1996 to 1999. He was at Envivio since its inception and led the Video Team for several years from 2000 to 2015.
Julien Le Tanou
bio
Julien Le Tanou is a principal engineer in video technology at MediaKind, Cesson-Sévigné, France, formerly Ericsson Media Solutions. In this role, he is responsible for leading research, innovation, and software development in video processing, compression, and associated encoding technologies. Since 2012, he has been with Envivio, France, in charge of research and algorithms design for Envivio’s SW video encoding solutions. Before Envivio, he was with Orange Labs, Cesson-Sévigné, and Dolby Laboratories, Burbank, CA, where he conducted research and development for the next generation of video coding standards. He received an MS degree in signal and image processing from Telecom Paris, Palaiseau, France, in 2010, and an MSc degree in computer and electrical engineering from Institut Superieure d’Electronique de Paris, Paris, France, in 2008.
Mederic Blestel
bio
Mederic Blestel is a principal video compression engineer with MediaKind, Cesson-Sévigné, France. He joined the Envivio Compression Team, Rennes, France, in 2011. As an experienced leader, his research interests focus on various aspects of image and video processing research. He has been heavily involved in various encoding projects and now mainly focuses on new technical trends in the video compression area. Before joining MediaKind, he contributed to Joint Photographic Experts Group (JPEG) and MPEG standardization within the Institute of Electronics and Telecommunications of Rennes (IETR) Laboratory, Rennes. He received an MSc degree from the Institut National des Sciences Appliquées Rennes (INSA Rennes), Rennes, in 2006.
Keyword(s): Hybrid Encoder, Quantization, Video Compression
Copyright: © 2022 SMPTE

Bibliographic Reference(s)

B. Li et al. “QP Determination by Lambda Value,” JCTVC-I0426, Geneva, May 2012. EXTERNAL
x265. [Online]. Available: https://bitbucket.org/multicoreware/x265 EXTERNAL
HEVC HM Reference Software. [Online]. Available: https://vcgit.hhi.fraunhofer.de/jvet/HM EXTERNAL
F. Bossen, “Common Test Conditions and Software Reference Configurations,” Document JCTVC-L1100, 12th Meeting ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Geneva, Switzerland, Jan. 2013. EXTERNAL
J. P. Henot et al., “High Efficiency Video Coding (HEVC): Replacing or Complementing Existing Compression Standards?” IEEE Int. Symp. Broadband Multimedia Syst. and Broadcasting (BMSB), London, U.K., pp. 1–6, Jun. 2013. EXTERNAL
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale Structural Similarity for Image Quality Assessment,” Proc. The Thrity-Seventh Asilomar Conf. on Signals, Syst. & Comput., 2003, vol. 2, pp. 1398–1402, Nov. 2003. EXTERNAL
M. Bichon et al., “Optimal Adaptive Quantization Based on Temporal Distortion Propagation Model for HEVC,” IEEE Trans. Image Process., vol. 28(11): 5419–5434, Nov. 2019. EXTERNAL
C. A. Segall and A. K. Katsaggelos, “Pre- and Post-Processing Algorithms for Compressed Video Enhancement,” Proc. Thirty-Fourth Asilomar Conf. on Signals, Syst. and Comput., pp. 1369–1373, Vol. 2, Oct. 2000. EXTERNAL
H. Schwarz et al, “CE7: Transform Coefficient Coding and Dependent Quantization” (Tests 7.1.2, 7.2.1), JVET-K0071, Ljubljana, Jul. 2018. EXTERNAL
J. Garrett-Glaser, “A Novel Macroblock-Tree Algorithm for High Performance Optimization of Dependent Video Coding in H.264/AVC,” Tech. Rep., Apr. 2009. EXTERNAL
The WebM Project, “VP9 Video Codec Summary.” Accessed: Jun. 2013. [Online]. Available: https://www.webmproject.org/vp9/ EXTERNAL
J. Sun et al, “Rate-Distortion Analysis of Dead-Zone Plus Uniform Threshold Scalar Quantization and Its Application— Part II: Two-Pass VBR Coding for H.264/AVC,” IEEE Trans. Image Process., vol. 22(1), pp. 215–228, Jan. 2013. EXTERNAL
S. J. P. Westen, R. L. Lagendijk, and J. Biemond, “Adaptive Spatial Noise Shaping for DCT Based Image Compression, 1996 Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Process. Conf. Proc., vol. 4, pp. 2124–2127 May 1996. EXTERNAL
Z. Li et al., “Toward A Practical Perceptual Video Quality Metric,” Netflix TechBlog, June 2016. [Online]. Available: https://netf lixtechblog.com/toward-a-practical-perceptualvideo-quality-metric-653f208b9652 EXTERNAL
Alliance for Open Media (AOM), “AV1 Bitstream and Decoding Process Specification,” 2019. Accessed: May 2019. [Online]. Available: https://aomedia.org/av1/specification/increase, resulting in advantageous local R-D tradeoffs. Additionally, we observe that for some transformed coefficient distributions, and given the context adaptive binary arithmetic coding (CABAC) context, a small QP decrease may result in almost no rate increment. This can be explained by two facts. First, the CABAC context may be better suited when changing QP, making the entropy coding of quantized coefficients more efficient. Second, a slight quantized coefficient bitrate increase can be compensated by the differential QP syntax bitrate decrease. These two cases generate a lower rate than foreseen that we name “rate-drop.” Overall, refining locally a set of QP candidates can, at the same time, be beneficial for both rate and distortion, without compromising any global RDO, and that is what LQR does. Finally, thanks to additional heuristics and optimizations (e.g., distortion estimation in the transform domain), the LQR implementation can be kept reasonable in terms of computation overhead, with an overall encoding runtime increase measured below 10% in a MediaKind optimized software HEVC encoder. The compression efficiency performance of the LQR algorithm combined with STAQ (RDSTQ), implemented into ×265 and using JCT-VC common test conditions, is summarized in Table 3. Complementary to the STAQ algorithm, the coding efficiency advantage is about 6% bitrate saving for the same PSNR- or SSIMbased quality. Conclusion By sharing an overview of the hybrid video coding scheme that holds for most modern video compression standards, we highlight the key role of the quantization in optimizing the video quality/bitrate tradeoff, as the (almost) only adjustable lossy processing step in any encoding system. We consequently detail the various levels of granularity and control points available for quantization optimization, most notably the block or CU-level QP adaptation. As practical examples, we introduce and share some insights on two differentiating quantization algorithms, namely STAQ and LQR. We show that a careful implementation of these two complementary algorithms can upgrade, by more than –25% in bitrate saving for the EXTERNAL
G. J. Sullivan and T. Wiegand, “Rate-Distortion Optimization for Video Compression,” IEEE Signal Process Mag., 15(6):74–90, Nov. 1998. EXTERNAL
Alliance for Open Media (AOM), “AV1 Reference Software.” Accessed: Jun. 2018. [Online]. Available: https://aomedia.google-source.com/aom/ EXTERNAL
IETF RFC 6386 [Active*]
T-REC-H.264.2003 — “Advanced Video Coding (AVC),” International Telecommunication Union-Telecommunication Recommendation (ITU-T) Recommendation H.264 and International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 14496-10, May 2003. EXTERNAL
T-REC-H.264.2013 — “High Efficiency Video Coding (HEVC),” International Telecommunication Union-Telecommunication Recommendation (ITU-T) Recommendation H.264 and International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 23008-2. Jan. 2013. EXTERNAL
T-REC-H.264.2020 — “High Efficiency Video Coding (HEVC),” International Telecommunication Union-Telecommunication Recommendation (ITU-T) Recommendation H.264 and International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 23008-2. Jul. 2020. EXTERNAL
T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL
T-REC-Q.6 — M. Karczewicz, Y. Ye, and I. Chong, “Rate Distortion Optimized Quantization,” ITU-T Q.6/SG16 VCEG, VQEG-AH21, Antalya, Turkey, Jan. 2008. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2022.3167782.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Michaël Ropert, Julien Le Tanou, and Mederic Blestel; Mastering Quantization is Key for Video Compression, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 5, June 2022); SMPTE, 2022. Available at https://doi.org/10.5594/JMI.2022.3167782

Snippet:

Michaël Ropert, Julien Le Tanou, and Mederic Blestel; Mastering Quantization is Key for Video Compression, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 5, June 2022); SMPTE, 2022. Available at https://doi.org/10.5594/JMI.2022.3167782

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Michaël Ropert, Julien Le Tanou, and Mederic Blestel; <cite>Mastering Quantization is Key for Video Compression</cite>, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 5, June 2022); SMPTE, 2022. Available at <a href="https://doi.org/10.5594/JMI.2022.3167782" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2022.3167782</a></span>

SMPTE's HTML Pub

Preview:

Michaël Ropert, Julien Le Tanou, and Mederic Blestel; Mastering Quantization is Key for Video Compression, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 5, June 2022); SMPTE, 2022
doi: 10.5594/JMI.2022.3167782
url: https://doi.org/10.5594/JMI.2022.3167782

Snippet:

<li>
Michaël Ropert, Julien Le Tanou, and Mederic Blestel; <cite id="bib-10-5594-jmi-2022-3167782">Mastering Quantization is Key for Video Compression</cite>, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 5, June 2022); SMPTE, 2022
<span class="doi">10.5594/JMI.2022.3167782</span>
</li>