Low-Complexity Quality Measurement for Real-Time Video Compression

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2022, No. 10, pp. 1–12
Abstract: A wide variety of video quality (VQ) metrics have been introduced over the past decades. VQ metrics are used in a range of applications, including video quality monitoring, encoder optimization, bitrate selection, and adaptive bitrate ladder configuration. While quality metrics have been developed that correlate well with human visual perception, they typically require high computational complexity. Other popular metrics are less complex, but are not accurate enough to base encoder decisions on. — In this paper, we discuss relevant quality metrics and their computational complexity. We address the difficulties around real-time VQ measurement, and determine which metrics can be used inside real-time encoders, to not only measure, but to actively control video quality. We include recent Machine Learning (ML) driven research, with a focus on the complexity introduced by ML techniques for video quality assessment. While these techniques provide benefits to improve VQ measurement and monitoring applications, we illustrate the fine line between accuracy of ML networks and the number of operations performed in these networks and their associated cost.
Publication Date: 2022-10-24
DOI: 10.5594/M001958
ISBN: Print: 978-1-61482-963-8
Link: https://doi.org/10.5594/M001958
Author(s): Jan De CockSynamedia
Axel De DeckerSynamedia
Sangar SivashanmugamSynamedia
Keyword(s): Video Quality, Quality Metrics, Real-time, Computational Complexity
Copyright: © 2022 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

D. Nandakumar, Y. Wu, H. Wei, and A. Ten-Ami, “On the accuracy of video quality measurement techniques”, 21st IEEE International Workshop on Multimedia Signal Processing (MMSP), 2019. EXTERNAL
D. Ramsook, A. Kokaram, N. O'Connor, N. Birkbeck, Y. Su and B. Adsumilli, “A differentiable estimator of VMAF for Video”, Picture Coding Symposium (PCS), 2021. EXTERNAL
Zhi Li, Kyle Swanson, Christos Bampis, Lukáš Krasula and Anne Aaron, “Toward a Better Quality Metric for the Video Community”, Netflix TechBlog, 2020. EXTERNAL
J. Søgaard, S. Forchhammer and J. Korhonen, “No-Reference Video Quality Assessment Using Codec Analysis”, IEEE Transactions on Circuits and Systems for Video Technology, 25(10), pp. 1637-1650, 2015. EXTERNAL
Pete Warden and Daniel Situnayake, “TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers”, 1st Edition, O’Reilly, 2020. EXTERNAL
Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, 13(4), 2004, 600–612. EXTERNAL
Chen Feng, Fan Zhang, and David R. Bull, “Deep VQA Based on a Novel Hybrid Training Methodology” EXTERNAL
S. Regunathan et al., "Efficient measurement of quality at scale in Facebook video ecosystem," Proc. SPIE 11510, Applications of Digital Image Processing XLIII, 2020. EXTERNAL
Li-Heng Chen, Christos G. Bampis, Zhi Li, Andrey Norkin, and Alan C. Bovik, “ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression”. EXTERNAL
A. Mittal, A. Moorthy, and A. Bovik. “No-Reference Image Quality Assessment in the Spatial Domain.” IEEE Transactions on Image Processing, 21(12), 2012, pp. 4695–4708. EXTERNAL
Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy and Megha Manohara, “Toward A Practical Perceptual Video Quality Metric”, Netflix TechBlog, 2016. EXTERNAL
Wang, Z., Simoncelli, E.P., Bovik, A.C. Multiscale Structural Similarity for Image Quality Assessment. 37th Asilomar Conference on Signals, Systems & Computers, 1398–1402, 2003. EXTERNAL
W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, B. Yu, “Definitions, methods, and applications in interpretable machine learning”, Proc. of the National Academy of Sciences, vol. 116 (44), Oct 2019. EXTERNAL
H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. on Computational Imaging, 3(1), pp. 47–57, Mar. 2017. EXTERNAL
M. Pinson, S. Wolf, “A new standardized method for objectively measuring video quality”, IEEE Transactions on broadcasting, 50(3), 2004, 312-322. EXTERNAL
J. You and J. Korhonen, “Deep Neural Networks for No-Reference Video Quality Assessment”, IEEE International Conference on Image Processing (ICIP), 2019, pp. 2349-2353. EXTERNAL
M. Saad, A. Bovik, and C. Charrier, “Blind image quality assessment: A natural scene statistics approach in the DCT domain,” IEEE Transactions on Image Processing, 21(8), pp. 3339–3352, 2012. EXTERNAL
H. Sheikh, and A. Bovik, “Image Information and Visual Quality”, IEEE Transactions on Image Processing, 15(2), 2006, 430–444. EXTERNAL
L. Murn, S. Blasi, A. Smeaton, N. O’Connor and M. Mrak, “Interpreting CNN For Low Complexity Learned Sub-Pixel Motion Compensation in Video Coding”, IEEE International Conference on Image Processing (ICIP), 2020. EXTERNAL
D. Varga, T. Szirányi, “No-reference video quality assessment via pretrained CNN and LSTM networks”, Signal, Image and Video Processing, vol.13(8), 2019. EXTERNAL
S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, “SSIM-motivated rate-distortion optimization for video coding,” IEEE Trans. Circuits Syst. Video Technol., 22(4), pp. 516–529, 2012. EXTERNAL
Z. Wang and A. Bovik, "Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures," IEEE Signal Processing Magazine, 26(1), 98-117, 2009. EXTERNAL
A. Mittal, R. Soundararajan, and A. Bovik. “Making a Completely Blind Image Quality Analyzer”, IEEE Signal Processing Letters, 22(3), 2013, pp. 209–212. EXTERNAL
S. Bosse, D. Maniry, K.-R. Mueller, T. Wiegand, and W. Samek, “Deep neural networks for no-reference and full-reference image quality assessment,” IEEE Transactions on Image Processing, 27(1), pp. 206–219, 2018. EXTERNAL
K.-S. Lu, A. Ortega, D. Mukherjee, and Y. Chen, “Perceptually inspired weighted MSE optimization using irregularity-aware graph Fourier transform,” IEEE Int. Conf. Image Process., 2020. EXTERNAL
M. Pinson, “Why No Reference Metrics for Image and Video Quality Lack Accuracy and Reproducibility”, IEEE Transactions on Broadcasting, 2022. EXTERNAL
Liu, Wentao & Duanmu, Zhengfang & Wang, Zhou. (2018). End-to-End Blind Quality Assessment of Compressed Videos Using Deep Neural Networks. 546-554. EXTERNAL
M. Torres Vega, D. Mocanu, S. Stavrou, A. Liotta, (2017), “Predictive no-reference assessment of video quality”. Signal Processing: Image Communication, 52, 20-32. EXTERNAL
S. Ling, Y. Baveye, D. Nandakumar, S. Sethuraman, and P. Le Callet, “Towards Better Quality Assessment of High-Quality Videos”, ACM Multimedia, 2020. EXTERNAL
Ding, K., Ma, K., Wang, S. et al., “Comparison of Full-Reference Image Quality Models for Optimization of Image Processing Systems”, Int J Comput Vis 129, 1258–1281 (2021). EXTERNAL
Y.-H. Huang, T.-S. Ou, P.-Y. Su, and H. Chen, “Perceptual rate-distortion optimization using structural similarity index as quality metric,” IEEE Trans. Circuits Syst. Video Technol., vol. 20(11), pp. 1614–1624, Nov. 2010. EXTERNAL
Lukáš Krasula, Karel Fliegel, Patrick Le Callet, Miloš Klíma, “On the accuracy of objective image and video quality models: New methodology for performance evaluation”, 8th International Conference on Quality of Multimedia Experience (QoMEX), 2016. EXTERNAL
M. Chen and A. Bovik, “Fast structural similarity index algorithm”, IEEE International Conference on Acoustics, Speech and Signal Processing, 2010, pp. 994-997. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001958.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Jan De Cock, Axel De Decker, and Sangar Sivashanmugam; Low-Complexity Quality Measurement for Real-Time Video Compression, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001958

Snippet:

Jan De Cock, Axel De Decker, and Sangar Sivashanmugam; Low-Complexity Quality Measurement for Real-Time Video Compression, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001958

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Jan De Cock, Axel De Decker, and Sangar Sivashanmugam; <cite>Low-Complexity Quality Measurement for Real-Time Video Compression</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at <a href="https://doi.org/10.5594/M001958" target="_blank" rel="noopener">https://doi.org/10.5594/M001958</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Jan De Cock, Axel De Decker, and Sangar Sivashanmugam; <cite id="bib-10-5594-m001958">Low-Complexity Quality Measurement for Real-Time Video Compression</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
<span class="doi">10.5594/M001958</span>
</li>