A Biologically-Inspired Approach to Making HDR Video Quality Assessment Easier

Bibliographic Reference(s)

• A Biological Framework for Perceptual Video Processing and Compression [Active]
• Perceptual Signal Coding for More Efficient Usage of Bit Codes [Active]
• A Quality Metric For High Dynamic Range [Active]
• x265, www.x265.org EXTERNAL
• Z. Wang et al., “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process., 13(4):600–612, Apr. 2004. EXTERNAL
• ITU-R Recommnedation BT.2390-0, “High Dynamic Range Television Production and International Programme Exchange,” International Telecommunication Union, Geneva, Switzerland, 2016. EXTERNAL
• M. C. Morrone and R. A. Owens, “Feature Detection from Local Energy,” Pattern Recogn. Lett., pp. 303–313, 1987. EXTERNAL
• P. Kovesi, “Image Features from Phase Congruency,” Videre: Journal of Computer Vision Research, 1(3):1–26, 1999. EXTERNAL
• T. O. Aydin, R. Mantiuk, and H.-P. Seidel, “Extending Quality Metrics to Full Luminance Range Images,” Human Vision and Electronic Imag. XIII, Bernice E. Rogowitz and Thrasyvoulos N. Pappas, Eds., Proc. SPIE, Vol. 6806, 2008. EXTERNAL
• Z. Wang and A. C. Bovik, “Mean Squared Error: Love it or Leave it? A New Look at Signal Fidelity Measures,” IEEE Signal Process. Mag., 26(1):98–117, Jan. 2009. EXTERNAL
• J. Froehlich et al., “Creating Cinematic Wide Gamut HDR-Video for the Evaluation of Tone Mapping Operators and HDR-Displays,” Proc. SPIE 9023, Digital Photography X, pp. 90230X–90230X, 2014. EXTERNAL
• R. Liu and R. Laganiere, “Phase Congruence Measurement for Image Similarity Assessment,” Pattern Recogn. Lett., 28(1): 166–172, 2007. EXTERNAL
• S. J. Daly, “Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity,” SPIE Human Vision, Visual Processing, and Digital Display III, vol. 1666, pp. 204–214, 1992. EXTERNAL
• D. J. Field, “Relationship between the Statistics of Natural Images and the Response Properties of Cortical Cells,” J. Opt. Soc. Am. A., 4(12):2379–2394, 1987. EXTERNAL
• Ffmpeg, www.ffmpeg.org EXTERNAL
• L. Zhang et al., “FSIM: A Feature Similarity Index for Image Quality Assessment,” IEEE Trans. Image Process., 20(8):2378–2386, 2011. EXTERNAL
• H. R. Sheikh and A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., 15(2):430–444, Feb. 2006. EXTERNAL
• C. Mantel, S. C. Ferchiu, and S. Forchhammer, “Comparing Subjective and Objective Quality Assessment of HDR Images Compressed with JPEG-XT,” in 16th Int. IEEE Workshop Multimed. Signal Process. (MMSP), 2014. EXTERNAL
• S. T. McCarthy and W. G. Owen, “Apparatus and Methods for Image and Signal Processing,” U.S. Patent No. 6014468 (2000), U.S. Patent No. 6360021 (2002), U.S. Patent No. 7046852 (2006), 1998. EXTERNAL
• Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multi-Scale Structural Similarity for Image Quality Assessment,” Proc. of the 37th IEEE Asilomar Conf. Signals, Syst. Comput., pp. 1398–1402, 2003. EXTERNAL
• J. Froehlich et al., “HdM-HDR-2014 Project,” http://www.hdm-stuttgart.de/~froehlichj/hdm-hdr-2014 EXTERNAL
• P. Hanhart, P. Kroshunov, and T. Ebrahimi, “Subjective Evaluation of Higher Dynamic Range Video,” Proc. SPIE 9217, Applications of Digital Image Processing XXXVII, 92170L, Sep. 2014. EXTERNAL
• Video Quality Experts Group, “Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment,” Mar. 2000. EXTERNAL
• The Mathworks, www.mathworks.com EXTERNAL
• P. Hanhart et al., “Benchmarking of Objective Quality Metrics for HDR Image Quality Assessment,” EURASIP J. Image Video Process., 2015 (1):39, 2015. EXTERNAL
• S. Winkler, Digital Video Quality: Vision Models and Metrics, John Wiley & Sons Ltd., England, 2005. EXTERNAL
• R. Mantiuk et al., “Predicting Visible Differences in Highdynamic Range Images: Model and its Calibration,” Proc. SPIE Human Vision and Electronic Imaging X, vol. 5666, 2005. EXTERNAL
• R. Liu and R. Laganiere, “On the Use of Phase Congruency to Evaluate Image Similarity,” Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Vol. 2, pp. II-II, 2006. EXTERNAL
• M. Rerabek et al., “Subjective and Objective Evaluation of HDR Compression,” Int. Workshop Video Process. Quality Metrics for Consumer Electronics - VPQM, Chandler, AZ, Feb. 2015. EXTERNAL
• M. Narwaria et al., “HDR-VDP-2.2: A Calibrated Method for Objective Quality Prediction of High-Dynamic Range and Standard Images,” J. Electron. Imag., 24(1):010501, 2015. EXTERNAL
• P. Kovesi, “Invariant Measures of Image Features from Phase Information,” Ph.D. dissertation, Dept. of Computer Science, University of Western Australia, 1996. for light HEVC compression, thus indicating significant distortion. These findings were consistent across the test sequences, thus indicating a systematic characteristic of HEVC compression rather than a content-dependent effect. Conclusion We have presented in this paper a set of video distortion metrics that could prove to be particularly useful for HDR video. The main motivating principle we presented was the “spatial detail” signal, which we used in two ways: (1) as a proxy for the original image data and (2) as a guide to the perceptually important features in HDR WCG video. The spatial detail signal is a condensed version of the original image that preserves the recognizable details in an image while discounting local luminance. From a practical standpoint, the important key characteristic of the spatial detail signal is that it is statistically stable having an expectably compact, peaked, biphasic, and symmetric distribution of values that is preserved across a wide range in images and video. Larger values—positive and negative—form a convenient guide to the kinds of features people tend to find significant. The spatial detail values nearer the zero midpoint of the distribution form a convenient guide to image regions that people would tend to classify as background or textural. Such feature and texture maps provide a stable framework in which to selectively investigate the perceptual potent highlights and dark details that are the hallmark of HDR video. Using the spatial detail as a proxy for the original image, we showed that correlation between the spatial detail signals of the uncompressed and compressed versions of HDR video was systematically affected by the aggressiveness of HEVC compression. By combining spatial detail correlation with our feature and texture assignment methods, we showed that texture correlation was impacted significantly more than feature correlation. Spatial detail correlation has several distinctions compared to established video quality metrics. It can be used selectively on bright and dark features and on textures. Moreover, spatial detail values are in the range of 0 to 1, which is more intuitive than the unbounded PSNR scale, while being a much more sensitive indicator than MS-SSIM over the range of compression levels typical of TV services. We have deliberately used the term “video distortion” instead of “video quality” throughout this paper. The main reason for doing so is that the metrics we proposed have not yet been compared to subjective test scores and thus may not yet be claimed to be a calibrated predictor of subjective video quality scores. In addition, it is not the intent of this paper to link the metrics we propose to subjective assessment. Rather, EXTERNAL
• P. Hanhart et al., “HDR Image Compression: A New Challenge for Objective Quality Metrics,” 16th Int. Workshop on QoMEX, 2014. EXTERNAL
• G. Valenzise, F. De Simone, P. Lauga, and F. Dufaux, “Performance Evaluation of Objective Quality Metrics for HDR Image Compression,” Proc. SPIE 9217, Applications of Digital Image Processing XXXVII, 2014. EXTERNAL
• S. T. McCarthy, “Theory and Practice of Perceptual Video Processing in Broadcast Encoders for Cable, IPTV, Satellite, and Internet Distribution,” Proc. SPIE 9014, Human Vision and Electronic Imaging XIX, pp. 901400–901400, 2014. EXTERNAL
• N. B. Nill and B. H. Bouzas, “Objective Image Quality Measure Derived from Digital Image Power Spectra,” Opt. Eng., 31(4):30–53, 1992. EXTERNAL
• R. Mantiuk et al., “HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions,” ACM Trans. Graph., 30(4):40.1–40.14, 2011. EXTERNAL
• M. C. Morrone and D. C. Burr, “Feature Detection in Human Vision: A Phase-Dependent Energy Model,” Proc. R. Soc. Lond. B Biol. Sci., 235(1280):221–245, 1988. EXTERNAL
• R-REC-BT.500-13 — Recommendation ITU-R BT.500-13, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” International Telecommunication Union, Radiocommunication Sector, Geneva, 2012. EXTERNAL
• ITU-R BT.2020-2 [Active]
• ITU-R BT.2100-2 [Active]
• T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL