Hybrid Images for Personalized Media Streaming Optimization

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2023, No. 10, pp. 19–20
Abstract: Efficiently transmitting media while maintaining or improving the quality of experience (QoE) poses a constant challenge. On the media playback side, there exists a variety of factors affecting the resulting QoE perceived by end users, such as device characteristics, viewing conditions, and individual visual acuity. Understanding the characteristics of playback-side context can provide opportunities for optimizing media delivery tailored to each user. — We present a novel approach to estimating viewers’ visible frequency ranges by creating hybrid images in a quantitatively controlled manner. The proposed approach, which can be applied to adaptive bitrate (ABR) streaming optimization, combines low- and high-frequency components of two different images providing multiscale image perception. The proposed method is designed to adjust the hybrid image size and the cutoff frequencies of lowpass and highpass filters of hybrid image synthesis according to the given frequency range under test. This method can be used to estimate the minimum video resolution for maintaining maximum QoE, in any viewing conditions and devices. — Our experimental results reveal a strong correlation between the minimum resolution obtained from hybrid image perception tasks and the traditional just noticeable difference (JND) of real images of different resolutions when presented side-by-side. These JND tasks exhibit a wider spread both in the measured minimum resolution and elapsed time to complete the tasks, potentially due to the complexity of the cognitive process and the variability caused by contents and subjects. In contrast, the hybrid image technique achieves more stable results with much less effort from subjects.
Publication Date: 2023-10-16
DOI: 10.5594/M002014
ISBN: Print: 978-1-61482-964-5
Link: https://doi.org/10.5594/M002014
Author(s): Doh-Suk KimDolby Laboratories, Inc
Scott DalyDolby Laboratories, Inc
Ludovic MalfaitDolby Laboratories, Inc
Neel ChaudhariDolby Laboratories, Inc
Jeffrey RiedmillerDolby Laboratories, Inc
Keyword(s): QoE, Hybrid Image, Playback-Side Context, Image Resolution, Viewing Distance, Streaming Optimization, Personalization
Copyright: © 2023 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

ISO.23009-1.2019 — International Organization for Standardization/International Electrotechnical Commision (ISO/IEC) 23009-1:2019, “Information Technology—Dynamic Adaptive Streaming Over HTTP (DASH)—Part 1: Media Presentation Description and Segment Formats (4th Ed.),” 2019. [Online]. Available: https://www.iso.org/standard/79329.html EXTERNAL
R. Bala, R. V. Klassen, and K. M. Braun, “Efficient and simple methods for display toneresponse characterization,” Journal of the Society for Information Display, vol. 15, no. 11, p. 947, 2007. EXTERNAL
S. Daly and H. C. Lee, Visual characterization using a display model, United States Patent: 5,754,222, 1996. EXTERNAL
J. Xue and C. W. Chen, “A study on perception of mobile video with surrounding contextual influences,” Proc. QoMEX, 2012, pp. 248–253. EXTERNAL
A. Oliva, A. Torralba, and P. G. Schyns, “Hybrid images,” ACM SIGGRAPH 2006, pp. 527– 532, 2006. EXTERNAL
S. Daly, T. Kunkel, G.-M. Su, and A. Choudhury, “Spatiochromatic and temporal natural image statistics modelling: Applications from display analysis to neural networks,” ei, vol. 35, no. 15, pp. 188-1-188–7, Jan. 2023. EXTERNAL
N. Ye, K. Wolski, and R. K. Mantiuk, “Predicting Visible Image Differences Under Varying Display Brightness and Viewing Distance,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA: IEEE, Jun. 2019, pp. 5429–5437. EXTERNAL
J. Gille, L. Arend, and J. O. Larimer, “Display characterization by eye: contrast ratio and discrimination throughout the grayscale,” presented at the Electronic Imaging 2004, B. E. Rogowitz and T. N. Pappas, Eds., San Jose, CA, Jun. 2004, p. 218. EXTERNAL
S. M. Thurman, P. G. Davey, K. L. McCray, V. Paronian, and A. R. Seitz, “Predicting individual contrast sensitivity functions from acuity and letter contrast sensitivity measurements,” Journal of Vision, vol. 16, no. 15, pp. 1–15, 2016. EXTERNAL
H. Levitt, “Transformed up-down methods in psychoacoustics,” J. Acoust. Soc. Amer., vol. 49, pp. 467–477, 1971. EXTERNAL
R. Pantos and W. May, "HTTP live streaming, RFC 8216," IETF, 2017. EXTERNAL
Yu Hu, Ming Ronnier Luo, Baiyue Zhao, and Mingkai Cao, “Developing a visual method to characterize displays,” presented at the IS&T Color Imaging Conferecne, 2019. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M002014.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Doh-Suk Kim, Scott Daly, Ludovic Malfait, Neel Chaudhari, and Jeffrey Riedmiller; Hybrid Images for Personalized Media Streaming Optimization, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at https://doi.org/10.5594/M002014

Snippet:

Doh-Suk Kim, Scott Daly, Ludovic Malfait, Neel Chaudhari, and Jeffrey Riedmiller; Hybrid Images for Personalized Media Streaming Optimization, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at https://doi.org/10.5594/M002014

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Doh-Suk Kim, Scott Daly, Ludovic Malfait, Neel Chaudhari, and Jeffrey Riedmiller; <cite>Hybrid Images for Personalized Media Streaming Optimization</cite>, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at <a href="https://doi.org/10.5594/M002014" target="_blank" rel="noopener">https://doi.org/10.5594/M002014</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Doh-Suk Kim, Scott Daly, Ludovic Malfait, Neel Chaudhari, and Jeffrey Riedmiller; <cite id="bib-10-5594-m002014">Hybrid Images for Personalized Media Streaming Optimization</cite>, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5
<span class="doi">10.5594/M002014</span>
</li>