TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 131, No. 4, pp. 50–58
Abstract: Viewing television (TV) with family and friends makes the experience more enjoyable. Recently, as more people watch TV programs on their mobile devices via the internet, opportunities to watch TV in groups are decreasing. We propose a companion robot to enhance human communication during the TV-viewing experience. The robot extracts keywords from the video, audio, and subtitle data of the TV program being watched, generates utterances from the keywords, and asks people questions related to the TV program. Viewers can chat with the robot, triggered by its questions. To confirm the utility of the robot, we conducted an experiment in which users watched TV with the robot. Results indicate that over 70% of the participants responded that the robot “promoted active conversations among people” and “created a more relaxed atmosphere.” Thus, our study demonstrates that a TV-watching robot can create novel and rich media experiences and stimulate communication among people.
Publication Date: 2022-05-01
DOI: 10.5594/JMI.2022.3160804
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2022.3160804
Author(s): Yuta Hagio
bio
Yuta Hagio received BE and ME degrees in computer science and engineering from Waseda University, Tokyo, Japan, in 2013 and 2015, respectively. He joined the Japan Broadcasting Corporation (NHK), Tokyo, in 2015, and since 2018 has worked at NHK Science and Technology Research Laboratories (STRL), Tokyo. He is currently engaged in research on the robots that watch TV with people. He is a member of Institute of Image Information and Television Engineers (ITE) Japan.
Marina Kamimura
bio
Marina Kamimura received a BE degree in design engineering from the Shibaura Institute of Technology University, Tokyo, Japan, in 2016. She joined the Japan Broadcasting Corporation (NHK) in 2016, and since 2019 has worked at NHK STRL. She is currently engaged in research on the robots that watch TV with people. She is a member of ITE Japan and Information Processing Society of Japan (IPSJ).
Yuta Hoshi
bio
Yuta Hoshi received the BE and ME degrees in electrical engineering from Niigata University, Niigata, Japan, in 2009 and 2011, respectively. He joined NHK in 2011, and since 2015 has worked at NHK STRL. He is currently engaged in research on the robots that watch TV with people. He is a member of ITE Japan and the Institute of Electronics, Information, and Communication Engineers (IEICE) Japan.
Yutaka Kaneko
bio
Yutaka Kaneko received BS and MS degrees in electrical engineering from the University of Electro-Communications, Tokyo, Japan, in 1988 and 1990, respectively. In 1990, he joined NHK and since 1994 has worked at NHK STRL, where he is engaged in research on robots that watch TV with people, networked broadcasting station systems, image processing, and motion picture coding. He is a member of ITE Japan, IEICE Japan, and IPSJ.
Masao Yamamoto
bio
Masao Yamamoto received the BE and ME degrees from the Tokyo University of Science, Tokyo, Japan, in 1994 and 1996, respectively. He joined NHK in 1996. Since 2015, he has been engaged in the research and development of video distribution platforms and robots for watching TV with people in NHK STRL. He currently serves as Head of STRL’s Internet Service Systems Research Division.
Keyword(s): Companion Robot, Dialogue System, Human–robot Interaction, Keyword Extraction, Utterance Generation
Copyright: © 2022 SMPTE

Bibliographic Reference(s)

H. Ogawa et al., “System Architecture for IoT Services With Broadcast Content,” Proc. IEEE Intl. Conf. Consumer Electronics, pp. 1–2, Jan. 2018. EXTERNAL
T. Y. Lin et al., “Microsoft COCO: Common Objects in Context,” European Conf. Computer Vision, pp. 740–755, 2014. EXTERNAL
K. Nakadai, H. G. Okuno, and T. Mizumoto, “Development, Deployment and Applications of Robot Audition Open Source Software HARK,” J. Robotics Mechatronics, 29(1):16–25, Feb. 2017. EXTERNAL
S. Nishimura, M. Kanbara, and N. Hagita, “Atmosphere Sharing With TV Chat Agents for Increase of User’s Motivation for Conversation,” Proc. Intl. Conf. Human-Computer Interaction, pp. 482–492, Jun 2019. EXTERNAL
Y. Hoshi et al., “Utterance Function for Companion Robot for Humans Watching Television,” Proc. IEEE Intl. Conf. Consumer Electronics, pp. 1–5, Jan. 2020. EXTERNAL
T. Kudo, “MeCab: Yet Another Part-of-Speech and Morphological Analyzer,” [in Japanese] 2006. [Online]. Available: http://taku910.github.io/mecab/ EXTERNAL
J. Wu et al., “A Chatbot AI Corresponded With TV Program (in Japanese)” ITE Annual Convention, 33B-2, 2019. EXTERNAL
United Nations, “Patterns and Trends in Household Size and Composition: Evidence From a United Nations Dataset,” 2019. EXTERNAL
Microsoft, Microsoft Azure. [Online]. Available: https://azure.microsoft.com/ EXTERNAL
T. Sato, T. Hashimoto, and M. Okumura, “Implementation of a Word Segmentation Dictionary Called Mecab-ipadic-NEologd and Study on How to Use it Effectively for Information Retrieval (in Japanese),” Annual Meeting Association Natural Language Processing, NLP2017–B6–1, 2017. EXTERNAL
T. Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality,” Proceedings of the 26th International Conf. Neural Information Processing Systems, 2:3111– 3119, 2013. EXTERNAL
S. Ren et al., “Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks,” IEEE Trans. Pattern Analysis and Machine Intelligence, 39(6):1137–1149, Jun. 2017. EXTERNAL
Amazon, Amazon Web Service. [Online]. Available: https://aws.amazon.com/ EXTERNAL
Vstone Company, Ltd., CommU. [Online]. Available: https://www.vstone.co.jp/products/commu/index.html EXTERNAL
Q. Hou et al., “Deeply Supervised Salient Object Detection With Short Connections,” IEEE Trans. Pattern Analysis and Machine Intelligence, 41(4):815–828, Apr. 2019. EXTERNAL
J. Goto et al., “A Spoken Dialogue Interface for TV Operations Based on Data Collected by Using WOZ Method,” Annual Meeting of the Association for Computational Linguistics, pp. 101– 104, Jul. 2003. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2022.3160804.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Yuta Hagio, Marina Kamimura, Yuta Hoshi, Yutaka Kaneko, and Masao Yamamoto; TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 4, May 2022); SMPTE, 2022. Available at https://doi.org/10.5594/JMI.2022.3160804

Snippet:

Yuta Hagio, Marina Kamimura, Yuta Hoshi, Yutaka Kaneko, and Masao Yamamoto; TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 4, May 2022); SMPTE, 2022. Available at https://doi.org/10.5594/JMI.2022.3160804

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Yuta Hagio, Marina Kamimura, Yuta Hoshi, Yutaka Kaneko, and Masao Yamamoto; <cite>TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication</cite>, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 4, May 2022); SMPTE, 2022. Available at <a href="https://doi.org/10.5594/JMI.2022.3160804" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2022.3160804</a></span>

SMPTE's HTML Pub

Preview:

Yuta Hagio, Marina Kamimura, Yuta Hoshi, Yutaka Kaneko, and Masao Yamamoto; TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 4, May 2022); SMPTE, 2022
doi: 10.5594/JMI.2022.3160804
url: https://doi.org/10.5594/JMI.2022.3160804

Snippet:

<li>
Yuta Hagio, Marina Kamimura, Yuta Hoshi, Yutaka Kaneko, and Masao Yamamoto; <cite id="bib-10-5594-jmi-2022-3160804">TV-Watching Robot: Toward Enriching Media Experience and Activating Human Communication</cite>, SMPTE Motion Imaging Journal ( Volume: 131, Issue: 4, May 2022); SMPTE, 2022
<span class="doi">10.5594/JMI.2022.3160804</span>
</li>