AI Image Analysis in Era of Short-Time Viewing

Metadata

Publisher: SMPTE
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Motion Imaging Jour.
Volume: 133, No. 2, pp. 20–27
Abstract: In the era when short videos are preferred, broadcasting stations have been enhancing momentums to distribute summary videos of broadcast content on social media. Therefore, we have developed automatic generation systems for news and program summary videos. Using a video summarization artificial intelligence (AI) that has learned the image composition and camerawork typical of important scenes, it is possible to automatically generate summary videos with a high quality close to videos edited by actual program production staff. Our systems include functions enabling users to easily modify the automatically generated summary videos. These systems have been on trial/practical use in many Japan Broadcasting Corp. (NHK) broadcasting stations. The generated summary videos are posted daily on social media. Furthermore, considering a program website is also important content that could boost viewer contact rates, we developed a support system for program website creation using an AI to extract thumbnails automatically. AI-generated thumbnail images can be used to develop program websites wit minimal effort. These technologies can streamline the production of internet content such as summary videos and program websites. Moreover, they will significantly boost internet deployments of various broadcasting programs.
Publication Date: 2024-04-01
DOI: 10.5594/JMI.2024/BZSK5411
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2024/BZSK5411
Author(s): Momoko Maezawa
Rei Endo
Takahiro Mochizuki
Keyword(s): Video Summarization, Thumbnail Extraction, Neural Network, Artificial Intelligence
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

How AI picks the highlights from Wimbledon fairly and fast, 2019. [Online]. Available: https://www.ibm.com/blog/how-ai-picks-the-highlights-from-wimbledon-fairly-and-fast/. EXTERNAL
AI-generated Highlights Tell the Story of the Masters, 2019. [Online]. Available: https://blog.video.ibm.com/ai-video-technology/ai-generated-highlights-tell-story-of-the-masters-2019/. EXTERNAL
Serie A agrees AI highlights deal with WSC Sports, 2022. [Online]. Available: https://www.sportbusiness.com/news/serie-a-agrees-ai-highlights-deal-with-wsc-sports/. EXTERNAL
Providing automatic highlight creation service using NTT DoCoMo Co., Ltd., and AI - Easily extract only the highlight scenes you want and reduce the burden of video editing, 2021. [Online]. Available: https://3x3exe.com/premier/news0220601-3/2. EXTERNAL
Hakuhodo DY Media Partners Develops an Automatic Drama Digest Video Generation System Together with the Tokyo University of Science: Trial Operation with a New Drama Program Broadcast on Tuesdays by TBS, 2019. [Online]. Available: https://www.inter-bee.com/2019/en/magazine/production/detail/?id=871. EXTERNAL
Y. Gao, T. Ahang, and H. Xiao, “Thematic video thumbnail selection,” in Proc. of ICIP, pp. 4333–4336, 2009. EXTERNAL
H.-C. Lian, X.-Q. Li, and B. Song, “Automatic video thumbnail selection,” in Proc. of Internat. Conf. on Multimedia Technol., pp. 242–245, 2011. EXTERNAL
Y. Song, M. Redi, J. Vallmitjana, and A. Jaimes, “To click or not to click: Automatic selection of beautiful thumbnails from videos,” in Proc. of ACM International Conference on Information and Knowledge Management, pp. 659–668, 2016. EXTERNAL
N. Arthurs, S. Birnbaum, and N. Gruver, “Selecting youtube video thumbnails via convolutional neural networks,” Technical Report, Stanford, 2017. EXTERNAL
N. Murray, L. Marchesotti, and F. Perronnin, “AVA: A large-scale database for aesthetic visual analysis,” in Proc. of CVPR, pp. 2408–2415, 2012. EXTERNAL
X. Jin, “ILGNet: Inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation,” IET Computer Vision, 13. 2: pp. 206–212, 2019. EXTERNAL
T. Mochizuki, Y. Kawai, N. Fujimori, M. Maezawa, R. Endo, Y. Asami, “Prototype of support system for news summary video production (in Japanese),” J. Instit. Image Inform. and Tel. Engin., 77 (2): pp. 262–271, 2023. EXTERNAL
D. Tran, L. Bourdev, R. Fergus, L. Torresani, M. Paluri, “Learning spatiotemporal features with 3D convolutional networks,” in Proc. of ICCV, pp. 4489–4497, 2015. EXTERNAL
Y. Kawai, R. Endo, N. Fujimori, T. Mochizuki, “Study of face recognition using deep neural network (in Japanese),” in Proc. of Forum of Information Technology (FIT), no. 3, H-003, pp. 103–104, 2019. EXTERNAL
Y. Kawai, R. Endo, N. Fujimori, T. Mochizuki, “Face detection using cascaded convolutional network (in Japanese),” in Proc. of ITE Annual Convention, no. 3, p. 22B-1, 2018. EXTERNAL
NHK NEWS WEB, [Online]. Available: https://www3.nhk.or.jp/news/movie.html. EXTERNAL
T. Mochizuki, Y. Kawai, “Program video summarization by fusion of multi-modal/timescale features using 1D-CNN (in Japanese),” in Proc. of IEICE General Conference, pp. D-12–26, 2022. EXTERNAL
Real Python, The Ultimate Guide to Speech Recognition [Online]. Available: https://realpython.com/python-speech-recognition/. EXTERNAL
M. Maezawa, R. Endo, N. Fujimori, T. Mochizuki, “A study on the selection of thumbnail images considering a genre of TV programs (in Japanese).” IEICE Technical Report, vol. 121, no. 155, PRMU2021–15, pp. 48–51, 2021. EXTERNAL
C. Szegedy, “Going deeper with convolutions,” in Proc. IEEE Cont. on Comput. Vis. and Pattern Recognotion. (CVPR), 2015. EXTERNAL
S. Loffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. 32nd International Conference on Machine Learning (ICML), pp. 448–456, 2015. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2024-BZSK5411.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Momoko Maezawa, Rei Endo, and Takahiro Mochizuki; AI Image Analysis in Era of Short-Time Viewing, MIJ 2024, Volume 133, Number 2 (pp. 20 to 27); SMPTE, 2024. Available at https://doi.org/10.5594/JMI.2024/BZSK5411

Snippet:

Momoko Maezawa, Rei Endo, and Takahiro Mochizuki; AI Image Analysis in Era of Short-Time Viewing, MIJ 2024, Volume 133, Number 2 (pp. 20 to 27); SMPTE, 2024. Available at https://doi.org/10.5594/JMI.2024/BZSK5411

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Momoko Maezawa, Rei Endo, and Takahiro Mochizuki; <cite>AI Image Analysis in Era of Short-Time Viewing</cite>, MIJ 2024, Volume 133, Number 2 (pp. 20 to 27); SMPTE, 2024. Available at <a href="https://doi.org/10.5594/JMI.2024/BZSK5411" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2024/BZSK5411</a></span>

SMPTE's HTML Pub

Preview:

Momoko Maezawa, Rei Endo, and Takahiro Mochizuki; AI Image Analysis in Era of Short-Time Viewing, MIJ 2024, Volume 133, Number 2 (pp. 20 to 27); SMPTE, 2024
doi: 10.5594/JMI.2024/BZSK5411
url: https://doi.org/10.5594/JMI.2024/BZSK5411

Snippet:

<li>
Momoko Maezawa, Rei Endo, and Takahiro Mochizuki; <cite id="bib-10-5594-jmi-2024-bzsk5411">AI Image Analysis in Era of Short-Time Viewing</cite>, MIJ 2024, Volume 133, Number 2 (pp. 20 to 27); SMPTE, 2024
<span class="doi">10.5594/JMI.2024/BZSK5411</span>
</li>