Enhancing Content Creation Workflows through Automatic Speech Recognition Techniques

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2023, No. 10, pp. 14–16
Abstract: This research provides a comprehensive review of prevailing Automatic Speech Recognition (ASR) methods and their profound impact on media production. Central to this exploration is the pivotal role of the transformer model, showcasing its unique self-attention mechanism ideal for tasks demanding comprehension of temporal relationships. We conduct an in- depth comparison, evaluating leading ASR models like Multilingual Machine Speech (MMS) from Meta, Whisper from OpenAI, and Google's Universal Speech Model (USM). Their performance is gauged against commercial services from tech giants such as Microsoft Azure, Amazon Web Services, and Google Cloud Platform. The paper covers aspects of ASR systems like voice activity detection, language identification, multilanguage support and the metrics used to characterize the accuracy of the systems. The research also identifies important challenges, like the dearth of data for some languages and the intricacies associated with linguistic nuances. Further, we discuss the role of ASR in media production, ranging from generating time-based captions to revolutionizing editing methodologies. Through a detailed breakdown of the ASR process, from audio preprocessing to postprocessing, this study endeavors to merge academic insights with practical applications, empowering media producers to leverage the immense capabilities of contemporary ASR technologies.
Publication Date: 2023-10-16
DOI: 10.5594/M002027
ISBN: Print: 978-1-61482-964-5
Link: https://doi.org/10.5594/M002027
Author(s): Randy FayanAvid Technology
Zahra MontajabiAvid Technology
Rob GonsalvesAvid Technology
Keyword(s): Artificial Intelligence, Machine Learning, Automatic Speech Recognition
Copyright: © 2023 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

ANSI/CTA-708-E S-2023 [Active]
EBU-TT-D (Tech 3380, EBU-TT-D Subtitling Distribution Format Version 1.0) from the European Broadcasting Union. EXTERNAL
"Specification of the EBU Subtitling data exchange format" (PDF). European Broadcasting Union. February 1991. Retrieved 10 March 2013. EXTERNAL
Jones, Graham. "Implementing closed captioning for DTV." Proc. Broadcast Engineering Conference, National Association of Broadcasters (NAB), Las Vegas, USA. 2004. EXTERNAL
"SCTE 21 2012 - STANDARD FOR CARRIAGE OF VBI DATA IN CABLE DIGITAL TRANSPORT STREAMS". Society of Cable Telecommunications Engineers. SCTE 21: 13. EXTERNAL
S. Team. "Silero VAD: pre-trained enterprise-grade voice activity detector (VAD), number detector and language classifier." (2021). EXTERNAL
Mehrish, A., Majumder, N., Bhardwaj, R., & Poria, S. (2023). A review of deep learning techniques for speech processing. Information Fusion. EXTERNAL
Park, Tae Jin, Naoyuki Kanda, Dimitrios Dimitriadis, Kyu J. Han, Shinji Watanabe, and Shrikanth Narayanan. "A review of speaker diarization: Recent advances with deep learning," Computer Speech & Language 72 (2022): 101317, 2022 EXTERNAL
M. Bhatia, et al., VoIP: An In-Depth Analysis - Voice Activity Detection. Cisco, October 2006 EXTERNAL
Graves, Alex & Mohamed, Abdel-Rahman & Hinton, Geoffrey. (2013). Speech Recognition with Deep Recurrent Neural Networks. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. 38. 10.1109/ICASSP.2013.6638947. EXTERNAL
B. Minixhofer, et al., "Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation," arXiv preprint arXiv:2305.18893, 2023 EXTERNAL
Park, Tae Jin, et al. "Auto-tuning spectral clustering for speaker diarization using normalized maximum eigengap." IEEE Signal Processing Letters 27 (2019): 381-385. EXTERNAL
R. Wicks and M Post, “A unified approach to sentence segmentation of punctuated text in many languages,” Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 2021 EXTERNAL
A. Radford, et al. Robust speech recognition via large-scale weak supervision. International Conference on Machine Learning. PMLR, 2023, Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., & Weber, G. (2019) EXTERNAL
"Timed Text (TT) Authoring Format 1.0 – Distribution Format Exchange Profile (DFXP)" EXTERNAL
STANDARD, SMPTE. "Timed Text Format." (2010) EXTERNAL
Abdel-Hamid, O., Mohamed, A., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014). Convolutional Neural Networks for Speech Recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22, 1533-1545. EXTERNAL
Y. Zhang, et al. (2023). Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages. ArXiv, abs/2303.01037. EXTERNAL
V. Pratap, et al., M. (2023). Scaling Speech Technology to 1,000+ Languages. ArXiv, abs/2305.13516. EXTERNAL
M. Heck, Automatic Language Identification for Natural Speech Processing Systems, The Department of Informatics Institute of Anthropomatics (IFA) Interactive Systems Laboratories (ISL), 2011 EXTERNAL
S. S. Stevens, J. Volkmann, and E. B. Newman. "A scale for the measurement of the psychological magnitude pitch." The journal of the acoustical society of America 8, no. 3 (1937): 185-190. EXTERNAL
G. Heigold, V. Vanhoucke, A. Senior, P. Nguyen, M. Ranzato, M. Devin, and J. Dean. Multilingual acoustic models using distributed deep neural networks. In Proc. of ICASSP, 2013. EXTERNAL
E. Sejdić, I. Djurović I., J. Jiang (2009). "Time-frequency feature representation using energy concentration: An overview of recent advances". Digital Signal Processing. 19 (1): 153–183. doi:10.1016/j.dsp.2007.12.004. EXTERNAL
Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. NIPS. [ATTENTION] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. CoRR, abs/1409.0473. EXTERNAL
Google. (2011) WebRTC. https://webrtc.org/ [Online, accessed August 2023] EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M002027.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Randy Fayan, Zahra Montajabi, and Rob Gonsalves; Enhancing Content Creation Workflows through Automatic Speech Recognition Techniques, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at https://doi.org/10.5594/M002027

Snippet:

Randy Fayan, Zahra Montajabi, and Rob Gonsalves; Enhancing Content Creation Workflows through Automatic Speech Recognition Techniques, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at https://doi.org/10.5594/M002027

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Randy Fayan, Zahra Montajabi, and Rob Gonsalves; <cite>Enhancing Content Creation Workflows through Automatic Speech Recognition Techniques</cite>, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5. Available at <a href="https://doi.org/10.5594/M002027" target="_blank" rel="noopener">https://doi.org/10.5594/M002027</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Randy Fayan, Zahra Montajabi, and Rob Gonsalves; <cite id="bib-10-5594-m002027">Enhancing Content Creation Workflows through Automatic Speech Recognition Techniques</cite>, SMPTE Meetings and Conferences ( October 2023); SMPTE, 2023, ISBN: 978-1-61482-964-5
<span class="doi">10.5594/M002027</span>
</li>