Toward Generalized Psychovisual Preprocessing for Video Encoding

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2021, No. 11, pp. 1–7
Abstract: Deep perceptual preprocessing has recently emerged as a new way to enable further bitrate savings across several generations of video encoders without breaking standards or requiring any changes in client devices. In this paper, we lay the foundations toward a generalized psychovisual preprocessing framework for video encoding and describe one of its promising instantiations that is practically deployable for video-on-demand, live, gaming and user-generated content. Results using state-of-the-art AVC, HEVC and VVC encoders show that average bitrate (BD-rate) gains of 11% to 17% are obtained over three state-of-the-art reference-based quality metrics (Netflix VMAF, SSIM and Apple AVQT), as well as the recently-proposed non-reference ITU-T p.1204 metric. The proposed framework on CPU is shown to be twice faster than x264 medium-preset encoding. On GPU hardware, our approach achieves 714fps for 1080p video (below 2ms/frame), thereby enabling its use in very-low latency live video or game streaming applications.
Publication Date: 2021-11-09
DOI: 10.5594/M001933
Link: https://doi.org/10.5594/M001933
Author(s): Aaron ChadhaiSIZE, London, UK
Mohammad Ashraful AnamiSIZE, London, UK
Matthias TrederiSIZE, London, UK
Ilya FadeeviSIZE, London, UK
Yiannis AndreopoulosiSIZE, London, UK
Keyword(s): Perceptual Optimization, Deep Neural Networks, Video Delivery
Copyright: © 2021 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

“VMAF, code repository.” online at: github.com/Netflix/vmaf. EXTERNAL
Z. Li. Video @Scale 2020: VMAF. At Scale Conference, 2020. EXTERNAL
A. Chadha, and Y. Andreopoulos, “Deep perceptual preprocessing for video coding,” Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 14852-14861, 2021. EXTERNAL
A. Chadha, et al., “Escaping the complexity-bitrate-quality barriers of video encoders via deep perceptual optimization,” Applications of Digital Image Processing XLIII. Vol. 11510. EXTERNAL
I. Katsavounidis, and L. Guo, “Video codec comparison using the dynamic optimizer framework,” Applications of Digital Image Processing XLI , 10752, 107520Q (2018). EXTERNAL
S. L. Regunathan, et al., “Efficient measurement of quality at scale in Facebook video ecosystem,” Applications of Digital Image Processing XLIII. Vol. 11510, 2020. EXTERNAL
“Apple AVQT presentation”, Apple Worldwide Developers Conference (WWDC) 2021. EXTERNAL
A. Raake, et al. “Multi-model standard for bitstream-, pixel-based and hybrid video quality assessment of UHD/4K: ITU-T P. 1204,” IEEE Access 8: 193020-193049 (2020). EXTERNAL
Q. Huynh-Thu, and M. Ghanbari, “Scope of validity of PSNR in image/video quality assessment,” Electronics Letters 44(13), 800–801 (2008). EXTERNAL
A. K. Venkataramanan, et al. “A hitchhiker’s guide to structural similarity,” IEEE Access 9 (2021): 28872-28896. EXTERNAL
S. Deng, et al. “VMAF based rate-distortion optimization for video coding,” Proc. IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2020. EXTERNAL
T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001933.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Aaron Chadha, Mohammad Ashraful Anam, Matthias Treder, Ilya Fadeev, and Yiannis Andreopoulos; Toward Generalized Psychovisual Preprocessing for Video Encoding, SMPTE Meetings and Conferences ( November 2021); SMPTE, 2021. Available at https://doi.org/10.5594/M001933

Snippet:

Aaron Chadha, Mohammad Ashraful Anam, Matthias Treder, Ilya Fadeev, and Yiannis Andreopoulos; Toward Generalized Psychovisual Preprocessing for Video Encoding, SMPTE Meetings and Conferences ( November 2021); SMPTE, 2021. Available at https://doi.org/10.5594/M001933

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Aaron Chadha, Mohammad Ashraful Anam, Matthias Treder, Ilya Fadeev, and Yiannis Andreopoulos; <cite>Toward Generalized Psychovisual Preprocessing for Video Encoding</cite>, SMPTE Meetings and Conferences ( November 2021); SMPTE, 2021. Available at <a href="https://doi.org/10.5594/M001933" target="_blank" rel="noopener">https://doi.org/10.5594/M001933</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Aaron Chadha, Mohammad Ashraful Anam, Matthias Treder, Ilya Fadeev, and Yiannis Andreopoulos; <cite id="bib-10-5594-m001933">Toward Generalized Psychovisual Preprocessing for Video Encoding</cite>, SMPTE Meetings and Conferences ( November 2021); SMPTE, 2021
<span class="doi">10.5594/M001933</span>
</li>