A Benchmark of Objective Quality Metrics for HLG-Based HDR/WCG Image Coding

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2020, No. 11, pp. 1–18
Abstract: In this work, we study the suitability of high dynamic range, wide color gamut (HDR/WCG) objective quality metrics to assess the perceived deterioration of compressed images encoded using the Hybrid Log-Gamma (HLG) method, which is the standard for HDR television. — Several image quality metrics have been developed to deal specifically with HDR content, although in previous work we showed that the best results (i.e., better matches to the opinion of human expert observers) are obtained by an HDR metric that consists simply in applying a given standard dynamic range metric, called visual information fidelity (VIF), directly to HLG-encoded images. — However, all these HDR metrics ignore the chroma components for their calculations, i.e., they just consider the luminance channel. For this reason, in the current work, we conduct subjective evaluation experiments in a professional setting using compressed HDR/WCG images encoded with HLG and analyze the ability of the best HDR metric to detect perceivable distortions in the chroma components, as well as the suitability of popular color metrics (including ΔITPR, which supports parameters for HLG) to correlate with the opinion scores. Our first contribution is to show that there is a need to consider the chroma components in HDR metrics, as there are color distortions that subjects perceive but that the best HDR metric fails to detect. Our second contribution is the surprising result that VIF, which utilizes only the luminance channel, correlates much better with the subjective evaluation scores than the metrics that do consider the color components.
Publication Date: 2020-11-10
DOI: 10.5594/M001925
Link: https://doi.org/10.5594/M001925
Author(s): Yasuko SugitoNHK, Tokyo, Japan
Trevor CanhamUniversitat Pompeu Fabra, Barcelona, Spain
Javier Vazquez-CorralUniversitat Pompeu Fabra, Barcelona, Spain
Marcelo BertalmioUniversitat Pompeu Fabra, Barcelona, Spain
Keyword(s): HDR, Wide Color Gamut, HLG, Image Compression, Objective Quality Metric
Copyright: © 2020 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

HDR and WCG Image Quality Assessment Using Color Difference Metrics [Active]
International Telecommunications Union – Radiocommunication (ITU-R), Report BT.2408-3, “Guidance for operational practices in HDR television production,” 07/2019. EXTERNAL
M. Rousselot, X. Ducloux, O. Le Meur and R. Cozot, "Quality Metric Aggregation for HDR/WCG Images," in Proc. 2019 IEEE International Conference on Image Processing (ICIP), pp. 3786-3790, 2019. EXTERNAL
A. Choudhury, and S. Daly, "Combining Quality Metrics for Improved HDR Image Quality Assessment," in Proc. 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 179-184, 2019. EXTERNAL
M. Narwaria, M. Perreira da Silva, and P. Le Callet, “HDR-VQM: An Objective Quality Measure for High Dynamic Range Video,” Signal Processing: Image Communication, vol. 35, pp. 46-60, Jul. 2015. EXTERNAL
E. François, J. Sole, J. Ström, and P. Yin “Common Test Conditions for HDR/WCG video coding experiments,” JCTVC-Z1020, 2017. EXTERNAL
L. Zhang, L. Zhang, X. Mou and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 8, pp. 2378-2386, Aug. 2011. EXTERNAL
G. Sharma, W. Wu, and E. N. Dalal, “The CIEDE2000 color difference formula: Implementation notes, supplementary test data, and mathematical observations,” Color Research & Application, vol. 30, no. 1, pp. 21-30, Feb. 2005. EXTERNAL
“Toward A Practical Perceptual Video Quality Metric,” Jul. 8, 2020. [Online] Available: https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652 EXTERNAL
Y. Sugito, and M. Bertalmío, "PERFORMANCE EVALUATION OF OBJECTIVE QUALITY METRICS ON HLG-BASED HDR IMAGE CODING," in Proc. 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 96-100, 2018. EXTERNAL
X. Zhang, and B.A. Wandell, "A spatial extension of CIELAB for digital color image reproduction." Journal of The Society for Information Display, vol. 5, pp. 61-63, 1997. EXTERNAL
M. Rousselot, O. Le Meur, R. Cozot, and X. Ducloux, “Quality assessment of HDR/WCG images using HDR uniform color spaces,” Journal of Imaging, vol. 5, no. 1, pp. 1-40, Jan. 2019. EXTERNAL
H.R. Sheikh and A.C. Bovik, “Image information and visual quality,” IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, Feb. 2006. EXTERNAL
Y. Sugito, and M. Bertalmío, "Practical Use Suggests a Re-evaluation of HDR Objective Quality Metrics," in Proc. 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pp. 1-6, 2019. EXTERNAL
J.A. Pytlarz, and E.G. Pieri, “How close is close enough?” in Proc. International Broadcasting Convention (IBC) 2017 Conference, pp.1-9, 2017. EXTERNAL
“Versatile Video Coding (VVC) | JVET,” Jul. 8, 2020. [Online], Available: https://jvet.hhi.fraunhofer.de/ EXTERNAL
“EBU Technology & Innovation — Zurich athletics,” Sep. 8, 2020. [Online], Available: https://tech.ebu.ch/testsequences/zurich_athletics EXTERNAL
M. Safdar, G. Cui, Y.J. Kim, and M.R. Luo, "Perceptually uniform color space for image signals including high dynamic range and wide gamut," Opt. Express, vol. 25, no.13, pp. 15131-15151, Jun. 2017. EXTERNAL
J. Froehlich, S. Grandinetti, B. Eberhardt, S. Walter, A. Schilling, and H. Brendel, “Creating cinematic wide gamut HDR video for the evaluation of tone mapping operators and HDR-displays,” in Proc. SPIE 9023, Digital Photography X, 90230X, 2014. EXTERNAL
M. Narwaria, R. Mantiuk, M. Perreira da Silva, and P. Le Callet. "HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images." Journal of Electronic Imaging, vol. 24, no. 1, 010501, Jan. 2015. EXTERNAL
M.D. Fairchild, “The HDR photographic survey,” in Proc. 15th Color and Imaging Conference (CIC 2007), pp. 233-238, 2007. EXTERNAL
“High Efficiency Video Coding (HEVC) | JCT-VC,” Jul. 8, 2020. [Online], Available: http://hevc.hhi.fraunhofer.de/ EXTERNAL
“HDR Reference Monitor ColorEdge Prominence CG3145 – EIZO,” Jul. 8, 2020. [Online], Available: https://www.eizo.com/products/coloredge/cg3145/ EXTERNAL
R-REC-BT.500-14 — International Telecommunication Union – Radiocommunication (ITU-R), Recommendation BT.500-14, “Methodologies for the Subjective Assessment of the Quality of Television Images,” Oct. 2019. EXTERNAL
ITU-R BT.709.6 [Active]
ITU-R BT.2020-2 [Active]
ITU-R BT.2100-2 [Active]
R-REC-BT.2124-0 — International Telecommunications Union—Radiocommunication (ITU-R), Recommendation BT.2124-0, “Objective Metric for the Assessment of the Potential Visibility of Colour Differences in Television,” Jan. 2019. EXTERNAL
T-REC-H.265.2017 — International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 23008-2:2017, “High Efficiency Coding and Media Delivery in Heterogeneous Environments—Part 2: High Efficiency Video Coding.” International Telecommunication Union – Telecommunication (ITU-T), Recommendation H.265, “High Efficiency Video Coding,” Feb. 2018. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001925.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Yasuko Sugito, Trevor Canham, Javier Vazquez-Corral, and Marcelo Bertalmio; A Benchmark of Objective Quality Metrics for HLG-Based HDR/WCG Image Coding, SMPTE Meetings and Conferences ( November 2020); SMPTE, 2020. Available at https://doi.org/10.5594/M001925

Snippet:

Yasuko Sugito, Trevor Canham, Javier Vazquez-Corral, and Marcelo Bertalmio; A Benchmark of Objective Quality Metrics for HLG-Based HDR/WCG Image Coding, SMPTE Meetings and Conferences ( November 2020); SMPTE, 2020. Available at https://doi.org/10.5594/M001925

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Yasuko Sugito, Trevor Canham, Javier Vazquez-Corral, and Marcelo Bertalmio; <cite>A Benchmark of Objective Quality Metrics for HLG-Based HDR/WCG Image Coding</cite>, SMPTE Meetings and Conferences ( November 2020); SMPTE, 2020. Available at <a href="https://doi.org/10.5594/M001925" target="_blank" rel="noopener">https://doi.org/10.5594/M001925</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Yasuko Sugito, Trevor Canham, Javier Vazquez-Corral, and Marcelo Bertalmio; <cite id="bib-10-5594-m001925">A Benchmark of Objective Quality Metrics for HLG-Based HDR/WCG Image Coding</cite>, SMPTE Meetings and Conferences ( November 2020); SMPTE, 2020
<span class="doi">10.5594/M001925</span>
</li>