AV1 Repetitive Film Grain Pattern Detection and Quality Assessment

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–14
Abstract: Film grain is a distinctive characteristic of analog film, renowned for its contribution to the cinematic look of videos. Preserving film grain integrity poses a challenge in video distribution, particularly in low-bitrate scenarios, as it greatly restricts compression gains. A V1 film grain synthesis framework addresses this by separating and estimating film grain during encoding, and using these estimates to reconstruct film grain during decoding. The choice of film grain block size, as an important parameter of the framework, significantly impacts reconstruction quality, with larger sizes potentially causing noticeable repetitive patterns, detracting from the viewing experience. In response, we propose a novel first-of-its-kind perceptual quality assessment method to detect and penalize these repetitive patterns efficiently, leveraging the characteristics of the human visual system. Our method normalizes input frames, finds its local similarities with the grain templates to localize repetitive patterns, and uses perceptual pooling to score film grain quality. Implemented in the frequency domain for efficiency, our method was validated through a subjective experiment we conducted with expert viewers. Our experiments show a high correlation between our method and human judgments of film grain quality, providing evidence for the effectiveness of the proposed method.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3022
ISBN: Electronic: 978-1-61482-965-2
Link: https://doi.org/10.5594/MOO/3022
Author(s): Armin Shafiee Sarvestani
Hojatollah Yeganeh
Keyword(s): Film Grain Quality Assessment, Film Grain Repetitive Pattern Detection, Auto-Regressive Film Grain Synthesis, Subjective Experiment
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, “Film grain synthesis technology for video applications,” 2023, Draft 4. EXTERNAL
J. C. K. Yan and D. Hatzinakos, “Signal-dependent film grain noise removal and generation based on higher-order statistics,” in Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics, 1997, pp. 77–81. EXTERNAL
J. Dai, “Film grain noise removal and synthesis in video coding,” in 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010, pp. 890–893. EXTERNAL
B. T. Oh, S. Lei, and C. C. J. Kuo, “Advanced film grain noise extraction and synthesis for high-definition video coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 12, pp. 1717–1729, 2009. EXTERNAL
Z. Ameur, “Deep-based film grain removal and synthesis,” IEEE Transactions on Image Processing, vol. 32, pp. 5046–5059, 2023. EXTERNAL
A. Newson, J. Delon, and B. Galerne, “A stochastic film grain model for resolution independent rendering,” Computer Graphics Forum, vol. 36, no. 8, pp. 684–699, 2017. EXTERNAL
A. Norkin and N. Birkbeck, “Film grain synthesis for av1 video codec,” in 2018 Data Compression Conference, 2018, pp. 3–12. EXTERNAL
Alliance for Open Media, “Scalable Video Technology for AV1 (SVT-AV1 Encoder),” https://gitlab.com/AOMediaCodec/SVT-AV1, Accessed: 2024–08-27. EXTERNAL
VideoLAN, “Dav1 d, av1 cross-platform open-source decoder,” https://code.videolan.org/videolan/dav1d, Accessed: 2024–02-16. EXTERNAL
S. J. Daly, “Visible differences predictor: an algorithm for the assessment of image fidelity,” in Human Vision, Visual Processing, and Digital Display III, Bernice E.Rogowitz, Ed. International Society for Optics and Photonics, 1992, vol. 1666, pp. 2–15, SPIE. EXTERNAL
C. J. Van den Branden Lambrecht and O. Verscheure, “Perceptual quality measure using a spatiotemporal model of the human visual system,” in Digital Video Compression: Algorithms and Technologies 1996. SPIE, 1996, vol. 2668, pp. 450–461. EXTERNAL
A. K. Moorthy and A. C. Bovik, “Perceptually significant spatial pooling techniques for image quality assessment,” in Human Vision and Electronic Imaging XIV. SPIE, 2009, vol. 7240, pp. 339–349. EXTERNAL
Video Quality Experts Group, “Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II,” VQEG, 2003. EXTERNAL
H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on image processing, vol. 15, no. 11, pp. 3440–3451, 2006. EXTERNAL
R-REC-BT.500-15 — International Telecommunications Union-Radiocommunication (ITU-R), Recommendation BT.500-15, “Methodologies for the subject assessment of the quality of television images,” May 2023. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3022.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Armin Shafiee Sarvestani and Hojatollah Yeganeh; AV1 Repetitive Film Grain Pattern Detection and Quality Assessment, MTS 2024, Article 22 (pp. 1 to 14); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3022

Snippet:

Armin Shafiee Sarvestani and Hojatollah Yeganeh; AV1 Repetitive Film Grain Pattern Detection and Quality Assessment, MTS 2024, Article 22 (pp. 1 to 14); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3022

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Armin Shafiee Sarvestani and Hojatollah Yeganeh; <cite>AV1 Repetitive Film Grain Pattern Detection and Quality Assessment</cite>, MTS 2024, Article 22 (pp. 1 to 14); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at <a href="https://doi.org/10.5594/MOO/3022" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3022</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Armin Shafiee Sarvestani and Hojatollah Yeganeh; <cite id="bib-10-5594-moo-3022">AV1 Repetitive Film Grain Pattern Detection and Quality Assessment</cite>, MTS 2024, Article 22 (pp. 1 to 14); SMPTE, 2024, ISBN: 978-1-61482-965-2
<span class="doi">10.5594/MOO/3022</span>
</li>