A Subjective Comparison of Broadcast and Unicast Transmission Impairments

Bibliographic Reference(s)

• P. Kline The Handbook of Psychological Testing, 2nd ed. Routledge: London, U.K., 2000. EXTERNAL
• T. N. Minhas, O. Gonzalez Lagunas, P. Arlos, and M. Fiedler, “Mobile Video Sensitivity to Packet Loss and Packet Delay Variation in Terms of QoE,” Proc. 19th International Packet Video Workshop (PV), Munich, pp. 83–88, 2012. EXTERNAL
• M. Mu, A. Mauthe, G. Tyson, and E. Cerqueira, “Statistical Analysis of Ordinal User Opinion Scores,” presented at the 9th The values we calculated for Cronbach’s alpha are shown in Table 2. It can be seen that the values of alpha for this experiment are much larger than 0.9, and hence we conclude that all of our experimental results are reliable. Conclusion We carried out subjective tests with 60 viewers over the age of 18, which were termed adult viewers, and with 570 young students under the age of 17, the majority being 10 or 11 years old, which were termed student viewers. We believe that the use of such a large population of viewers, with such a range of ages, makes this a very rare, if not unique, experiment. The aim of the experiment was to understand the relative annoyance caused by various transmission impairments when viewing television content, and to study the impact of the type of screen on which the content is viewed, considering viewing on tablets and on a TV, and to study the impact of the age of the viewer. We found a significant level of consistency from the overall results of our experiments. We found the following results: that interruptions are the most annoying impairment that was tested, multiple short interruptions are more annoying than a single longer interruption of the same duration, and it is more annoying when the interruption causes some of the content not to be presented, rather than just a pause. It was observed that dropping to a very low quality for a short period of time and dropping to a medium quality for a longer period are more annoying while watching on a TV than on a tablet. Compared to the adult viewers, younger viewers are more sensitive to impairments when viewing on a tablet while older viewers are more sensitive to impairments when viewing on a TV. A reasonably high level of similarity exists between the opinions of the adult viewers and the student viewers, suggesting to some extent that the relative annoyance caused by different impairments is independent of age. The experiment and its results also show that students, as young as 10 or 11 years of age, can be used as viewers in video subjective testing if the test is designed appropriately, which in this experiment meant using short test sessions of 6 min each. We hope that the results obtained will influence the design of future television content delivery services, so that if transmission impairments do occur, the service is designed to minimize the annoyance caused. We also hope that we motivate other researchers to include young students in their experiments, not only because their views are valid, but also to give them direct experience of work in the technology industry and hence to encourage them to study STEM9 subjects to achieve a rewarding career in the industry. EXTERNAL
• L. J. Cronbach, “Coefficient Alpha and the Internal Structure of Tests,” Psychometrika, 16:297–334, 1951. EXTERNAL
• British Telecommunication plc, BT TV. [Online]. Available from: http://sport.bt.com/ EXTERNAL
• L. Pozueco et al., “Subjective Video Quality Evaluation of Different Content Types Under Different Impairments,” New Rev. Hypermedia Multimedia, 23(1):1–28, 2017. EXTERNAL
• J. Nightingale, Q. Wang, C. Grecos, and S. Goma, “The Impact of Network Impairment on Quality of Experience (QoE) in H.265/HEVC Video Streaming,” IEEE Trans. Consum. Electron., 60(2):242–250, 2014. EXTERNAL
• D. Ghadiyaram, A. C. Bovik, H. Yeganeh, R. Kordasiewicz, and G. Michael, “Study of the Effects of Stalling Events on the Quality of Experience of Mobile Streaming Videos,” presented at the 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, GA. EXTERNAL
• STEM: Science, Technology, Engineering and Mathematics. [Online]. Available from: https://www.stem.org.uk/ EXTERNAL
• T. De Pessemier, K. De Moor, W. Joseph, L. De Marez, and L. Martens, “Quantifying the Influence of Rebuffering Interruptions on the User’s Quality Of Experience During Mobile Video Watching,” IEEE Trans. Broadcast., 59(1):47–61, 2013. EXTERNAL
• R-REC-BT.500-13 — Recommendation ITU-R BT.500-13, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” International Telecommunication Union, Radiocommunication Sector, Geneva, 2012. EXTERNAL
• ITU-T REC H.264 [Active]
• ITU-T REC H.265 [Active]
• T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL
• T-REC-P.1203 — International Telecommunication Union - Telecommunication (ITU-T), Recommendation P.1203 (10/17), “Parametric Bitstream-Based Quality Assessment of Progressive Download and Adaptive Audiovisual Streaming Services Over Reliable Transport,” Oct. 2017. EXTERNAL