360-Degree Video Streaming and its Subjective Quality

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2017, No. 10, pp. 1–23
Abstract: Traditional challenges for deploying end-to-end streaming systems are made harder when considering 360-degree media content. One of these challenges relates to the lack of commonly accepted standardized methodologies for subjective 360-degree video quality assessment, especially oriented towards streaming services. The contribution of this paper falls in the area of subjective assessment of 360-degree video. — From traditional standardized test methodologies originally designed for 2D/3D video, we tailored a methodology more oriented towards Virtual Reality (VR) streaming services. The methodology inherits a lot from existing ITU standards for video subjective quality evaluation. The additions incorporate the special properties of 360-video, namely omnidirectionality, as opposed to traditional video. — With this goal in mind, a new metric called Similarity Ring Metric (SRM) is introduced. It measures the degree of similarity in watching patterns of a single subject or between different subjects for several subjective assessment tests. This metric enables an inclusion or rejection criteria for test results in subjective assessment sessions. We also present visual fatigue results related to a subjective quality experiment of 360-degree video.
Publication Date: 2017-10-23
DOI: 10.5594/M001758
ISBN: Print: 978-1-61482-959-1
Link: https://doi.org/10.5594/M001758
Author(s): Igor D.D. CurcioDigital Media Laboratory, Nokia Technologies, Tampere, Finland
Henri ToukomaaDigital Media Laboratory, Nokia Technologies, Tampere, Finland
Deepa NaikDepartment of Signal Processing, Tampere University of Technology, Tampere, Finland
Keyword(s): 360° Video, Virtual Reality Streaming, Subjective Quality Assessment, Test Methodology
Copyright: © 2017 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

ISO.23009-1 — ISO/IEC 23009-1, “Information Technology—Dynamic Adaptive Streaming over HTTP (DASH)—Part 1: Media Presentation Description and Segment Formats,” 2016. EXTERNAL
S. Ishihara, The Series of Plates Designed as a test for Colour Deficiency. Concise Edition. Tokyo: Kanehara Trading, 1997. EXTERNAL
E. Upenik, M. Rerabek and T. Ebrahimi, “A testbed for subjective evaluation of omnidirectonal visual content”, 32nd Picture Coding Symposium (PCS), 4-7 December 2016, Nuremberg, Germany. EXTERNAL
R. Schatz, et al., “Towards Subjective Quality of Experience Assessment for Omnidirectional Video Streaming”, IEEE 9th Int. Conf. on QoMEX, Erfurt, Germany, (31 May 2017). EXTERNAL
International Telecommunications Union – Standardization (ITU-T), “Test plan for subjective assessment for VR video quality”, VQEG, IMG116, 11 October 2016. EXTERNAL
International Telecommunications Union – Standardization (ITU-T), “Test plan for evaluation of video quality models for use with stereoscopic three-dimensional television content”, VQEG, 2012. EXTERNAL
I.D.D. Curcio, “Similarity Ring Metric (SRM) for Subjective Evaluation of 360-Degree Video”, ISO/IEC JTC1/SC29/WG11 MPEG #119, m41303, Torino, Italy, (July 2017). EXTERNAL
TNO, Test for Stereoscopic Vision, Nieuwegein: Lameris Ootech, 1972. EXTERNAL
M. Yu, H. Lakshman and B. Girod, “A framework to evaluate omnidirectional video coding schemes”, IEEE International symposium on mixed and augmented reality, Fukuoka, Japan. (29 Sept.-3 Oct. 2015). EXTERNAL
C. Ozcinzar, A.D. Abreu and A. Smolic, “Viewport-aware adaptive 360° video streaming using tiles for virtual reality”, IEEE International Conference on Image Processing 2017, Beijing, China. (17-20 September 2017). EXTERNAL
I.D.D. Curcio, H. Toukomaa and D. Naik, “Bandwidth Reduction of Omnidirectional Viewport-Dependent Video Streaming via Subjective Quality Assessment”, ACM International Workshop on Multimedia Alternate Realities within the ACM Multimedia 2017 Conference, 27 October 2017, Mountain View, CA, U.S.A. EXTERNAL
D.S. Hands, “Temporal characteristics of forgiveness effect”, Electronics Letters, 37, pp. 752-754, 2001. EXTERNAL
C-L. Fan, et al., “Fixation Prediction for 360° Video Streaming in Head-Mounted Virtual Reality”, ACM 27th Workshop on NOSSDAV ’17, Taipei, Taiwan, (20-23 Jun 2017), 67-72. EXTERNAL
C. Ozcinzar, A.D. Abreu and A. Smolic, “Look around you: Saliency maps for omnidirectional images in VR applications View Document”, Ninth International Conference on Quality of Multimedia Experience (QoMEX), Erfurt, Germany. (31 May-2 June 2017). EXTERNAL
A. Zare, et al., “HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications”, In Proceedings of the 2016 ACM on MM '16, ACM, New York, NY, USA, 601-605. EXTERNAL
E. Uenik, M. Rerabek and T. Ebrahimi, “On the performance of objective metrics for omnidirectional visual content”, Ninth International Conference on Quality of Multimedia Experience (QoMEX), Erfurt, Germany. (31 May-2 June 2017). EXTERNAL
S. Shi, K. Nahrstedt and R. Campbell. “Distortion over Latency: Novel Metric for Measuring Interactive Performance in Remote Rendering Systems”, ICME, Barcelona, Spain, (11-15 Jul 2001). EXTERNAL
R-REC-BT.500-13 — Recommendation ITU-R BT.500-13, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” International Telecommunication Union, Radiocommunication Sector, Geneva, 2012. EXTERNAL
R-REC-BT.2021-1 — International Telecommunication Union - Radiocommunication (ITU-R), Recommendation BT.2021-1, “Subjective Methods for the Assessment of Stereoscopic 3DTV Systems,” Feb. 2015. EXTERNAL
T-REC-P.800.2 — International Telecommunication Union - Standardization (ITU-T), Recommendation P.800.2, “Mean Opinion Score Interpretation and Reporting,” Jun. 2016. EXTERNAL
T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL
T-REC-P.913 — International Telecommunication Union - Standardization (ITU-T), Recommendation P.913, “Methods for the Subjective Assessment of Video Quality, Audio Quality and Audiovisual Quality of Internet Video and Distribution Quality Television in Any Environment,” Mar. 2016. EXTERNAL
T-REC-P.915 — International Telecommunication Union - Standardization (ITU-T), Recommendation P.915, “Subjective Assessment Methods for 3-D Video,” Mar. 2016. EXTERNAL
T-REC-P.916 — International Telecommunication Union - Standardization (ITU-T), Recommendation P.916, “Information and Guidelines for Assessing and Minimizing Visual Discomfort and Visual Fatigue from 3-D Video,” Mar. 2016. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001758.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Igor D.D. Curcio, Henri Toukomaa, and Deepa Naik; 360-Degree Video Streaming and its Subjective Quality, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at https://doi.org/10.5594/M001758

Snippet:

Igor D.D. Curcio, Henri Toukomaa, and Deepa Naik; 360-Degree Video Streaming and its Subjective Quality, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at https://doi.org/10.5594/M001758

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Igor D.D. Curcio, Henri Toukomaa, and Deepa Naik; <cite>360-Degree Video Streaming and its Subjective Quality</cite>, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at <a href="https://doi.org/10.5594/M001758" target="_blank" rel="noopener">https://doi.org/10.5594/M001758</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Igor D.D. Curcio, Henri Toukomaa, and Deepa Naik; <cite id="bib-10-5594-m001758">360-Degree Video Streaming and its Subjective Quality</cite>, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1
<span class="doi">10.5594/M001758</span>
</li>