Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 132, No. 3, pp. 61–66
Abstract: The addition of haptic feedback as well as visual and audio information can help make live sports broadcasts more immersive. However, it is difficult to add haptic information to the video content in realtime. Thus, we developed a system that analyzes sports videos with low computational cost and displays haptic information depending on the results. The system measures the intensity value of athletes’ motion and the timings of specific actions from input video in realtime. The haptic chair and handheld devices are actuated by the trigger from the video analysis. The system provides viewers with haptic stimuli synchronized with audiovisual information. The haptic stimuli from the devices enhance viewers’ feelings of immersion while watching live sports broadcasts at public viewing events.
Publication Date: 2023-04-01
DOI: 10.5594/JMI.2023.3249359
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2023.3249359
Author(s): Masaki Takahashi
bio
Masaki Takahashi received BS and MS degrees in engineering from Keio University, Tokyo, Japan, in 1997 and 1999, respectively, and a PhD degree from the Graduate University of Advanced Studies, Hayama-cho, Kanagawa, Japan, in 2012. He joined Japan Broadcasting Corporation (NHK), Shibuya-ku, Tokyo, in 1999, and he has been working at the Science and Technology Research Laboratories, Setagaya-ku, Tokyo, since 2002. His research interests include visual object tracking and content production technology. He was a visiting researcher at the University of British Columbia, Vancouver, BC, Canada, in 2012.
Taichi Ishiwatari
bio
Taichi Ishiwatari received BE and ME degrees in electrical engineering and bioscience from Waseda University, Shinjuku-ku, Tokyo, Japan, in 2015 and 2017, respectively. After his graduation, he joined NHK and has been working at the Science and Technology Research Laboratories since 2019. His research interests include machine learning and natural language processing.
Masanori Sano
bio
Masanori Sano received BE and ME degrees in electronic engineering from Waseda University and a PhD degree in informatics from the Graduate University for Advanced Studies. He joined NHK in 1994, and since 1997 he has been with Science and Technology Research Laboratories. His research interests include information extraction from media, information integration, metadata production system, and universal services. He has been involved with activities at International Telecommunication Union (ITU), Moving Picture Experts Group (MPEG), European Broadcasting Union (EBU), Asia-Pacific Broadcasting Union (ABU), and Association of Radio Industries Businesses (ARIB).
Keyword(s): Haptics, Live Sports, Machine Learning
Copyright: © 2023 SMPTE

Bibliographic Reference(s)

S. J. Lederman and R. L. Klatzky, “Haptic Perception: A Tutorial,” Attention Perception Psychophys., 71(7):1439–1459, Oct. 2009. EXTERNAL
VisionPose. Accessed: 2019. [Online]. Available: https://www.next-system. com/visionpose. EXTERNAL
J. Li, C. Wang, H. Zhu, Y. Mao, H.-S. Fang, and C. Lu, “CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark,” Proc. of IEEE/CVF Conf. on Comput. Vision and Pattern Recognit. (CVPR), 2019. EXTERNAL
R. Takagi, N. Hashida, and H. Takeuchi, “Usability and Design Guideline for Designing Single Handle Faucet’s and Handle Shape,” Proc. of AHFE Int. Conf. on Usability and User Experience and Human Factors and Assistive Technol., pp. 89–98, 2018. EXTERNAL
H. Chen et al., “Frequency Domain Compact 3D Convolutional Neural Networks,” Proc. of the Int. Conf. on Comput. Vision and Pattern Recognit. (CVPR), pp. 1641–1650, 2020. EXTERNAL
C. B. Santiago, A. Sousa, M. L. Estriga, L. P. Reis, and M. Lames, “Survey on Team Tracking Techniques Applied to Sports,” Proc. of Int. Conf. on Auton. and Intell. Syst., pp. 1–6, 2010. EXTERNAL
J. Cha, Y. S. Ho, Y. Kim, and J. Ryu, “A Framework for Haptic Broadcasting,” IEEE Multimedia, 16(3):16–27, Jul. 2009. EXTERNAL
M. V. Valueva, N. N. Nagornov, P. A. Lyakhov, G. V. Valuev, and N. I. Chervyakov, “Application of the Residue Number System to Reduce Hardware Costs of the Convolutional Neural Network Implementation.” Math. Comput. Simul., 177:232–243, May. 2020. EXTERNAL
A. G. Thomas, “Virtual Graphics for Broadcast Production,” IEEE Comput., 6(7):42–47, 2019. EXTERNAL
M. Kristan et al., “The Eighth Visual Object Tracking VOT2020 Challenge Results,” Proc. of the European Conf. on Comput. Vision Workshops (ECCV Workshops), pp. 547–601, 2020. EXTERNAL
D. Wang, K. Ohnishi, and W. Xu, “Multimodal Haptic Display for Virtual Reality: A Survey,” IEEE Trans. Ind. Electron., 67(1):610–623, 2020. EXTERNAL
A. R. Ahead, T. Jie, H. Kim, and S. Ishikawa, “Motion History Image: Its Variants and Applications,” Mach. Vision and Appl., 23(2):255–281, Mar. 2010. EXTERNAL
J. Cha, J. Ryu, S. Kim, S. Eom, and B. Ahz, “Haptic Interaction in Realistic Multimedia Broadcasting,” Proc. of Pacific Rim Conf. on Adv. in Multimedia Inf. Process., 128: 482–490, 2004. EXTERNAL
M. Takahashi, M. Azuma, T. Handa, T. Ishiwatari, M. Sano, and Y. Yamanouchi, “Public Viewing With Haptic Devices Synchronized With Live Sports Broadcasts,” Proc. of the ACM SIGGRAPH, Posters, No. 25, 2021, doi:10.1145/3450618.3469135. EXTERNAL
J. P. Bresciani, K. W. Drewing, and M. O. Ernst, “Human Haptic Perception and the Design of Haptic-Enhanced Virtual Environments,” in The Sense of Touch and Its Rendering, STAR, Springer, vol. 45, pp. 61–106, 2008. EXTERNAL
Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), 43(1):172–186, Jan. 2019. EXTERNAL
C. N. Phyo, T. T. Zin, and P. Tin, “Skeleton Motion History Based Human Action Recognition Using Deep Learning,” Proc. of the IEEE/CVF Conf. on Comput. Vision and Pattern Recognit. EXTERNAL
M. Takahashi, M. Azuma, T. Ishiwatari, T. Handa, M. Sano, and Y. Yamanouchi, “Haptic Perception System Synchronized With Live Broadcasts,” Proc. of NAB Broadcast Eng. and Inf. Technol. Conf., TV Enhancement, 2021. EXTERNAL
I. Rakkolainen, A. Sand, and R. Raisamo, “A Survey of Mid-Air Ultrasonic Tactile Feedback,” Proc. of IEEE Int. Symp. on Multimedia (ISM2019), pp. 94–98, 2019. EXTERNAL
K. Fukushima, “Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position,” Biol. Cybern., 36:193–202, Apr. 1980. EXTERNAL
A. Lecuyer, “Simulating Haptic Feedback Using Vision: A Survey of Research and Applications of Pseudo-Haptic Feedback,” Presence, 18(1):39–53, Feb. 2009. EXTERNAL
K. Hayakawa et al., “An Empirical Practice of Live Feeling of a Professional Basketball Game With Haptic Transmissions,” Proc. of Eurohaptics 2018, 2018, ID.1122. EXTERNAL
A. Bulat, J. Kossaifi, G. Tzimiropoulos, and M. Pantic, “Toward Fast and Accurate Human Pose Estimation via Soft-Gated Skip Connections,” Proc. of IEEE Int. Conf. on Autom. Face & Gesture Recognit. (FG), 2020. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2023.3249359.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Masaki Takahashi, Taichi Ishiwatari, and Masanori Sano; Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3249359

Snippet:

Masaki Takahashi, Taichi Ishiwatari, and Masanori Sano; Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3249359

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Masaki Takahashi, Taichi Ishiwatari, and Masanori Sano; <cite>Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at <a href="https://doi.org/10.5594/JMI.2023.3249359" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2023.3249359</a></span>

SMPTE's HTML Pub

Preview:

Masaki Takahashi, Taichi Ishiwatari, and Masanori Sano; Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023
doi: 10.5594/JMI.2023.3249359
url: https://doi.org/10.5594/JMI.2023.3249359

Snippet:

<li>
Masaki Takahashi, Taichi Ishiwatari, and Masanori Sano; <cite id="bib-10-5594-jmi-2023-3249359">Haptic Presentation System Synchronized With Live Sports Broadcasts at Public Viewing Events</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023
<span class="doi">10.5594/JMI.2023.3249359</span>
</li>