AI Translation and Dubbing with Emotional Accuracy

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–16
Abstract: Integrating artificial intelligence (AI) into media has led to significant advancements in translation and dubbing, areas previously limited by technological constraints. However, initial Text-to-Speech (TTS) technologies were primarily developed for voice interfaces rather than cinematic dubbing, where emotional integrity is paramount. As a result, these early solutions are insufficient for the high emotional demands of film production. This paper examines AI's potential to enhance emotional accuracy in media translation and dubbing, focusing on the critical challenge of interlingual emotion transfer. Traditional methods to date do not allow for the preservation of the emotional integrity of content, leading to reduced audience engagement and cultural authenticity. Our research aims to develop and apply AI models that accurately detect and reproduce emotional cues across languages, improving both engagement and authenticity. While promising results have been achieved, challenges remain in the areas of cultural context and real-time processing, which this paper addresses in detail.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3043
ISBN: [object Object]
Link: https://doi.org/10.5594/MOO/3043
Author(s): Denys Krasnikov
Alexander Konovalov
Keyword(s): AI Translation, Dubbing, Emotional Accuracy, Natural Language Processing, Deep Learning, Speaker Diarization, AI Ethics, Cultural Adaptation, Responsible AI, Interlingual Transfer, Video Production, Automatic Speech Recognition, Machine Learning, Vidby Ag, Real-time Processing, AI-driven Media, Interlingual Emotion Transfer, Voice Cloning, Text-to-Speech, Lip Sync, Accessibility in Media, Large Language Models (LLMs)
Copyright: © 2024 SMPTE

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3043.json

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Denys Krasnikov and Alexander Konovalov; AI Translation and Dubbing with Emotional Accuracy, MTS 2024, Article 32 (pp. 1 to 16); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3043

Snippet:

Denys Krasnikov and Alexander Konovalov; AI Translation and Dubbing with Emotional Accuracy, MTS 2024, Article 32 (pp. 1 to 16); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3043

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Denys Krasnikov and Alexander Konovalov; <cite>AI Translation and Dubbing with Emotional Accuracy</cite>, MTS 2024, Article 32 (pp. 1 to 16); SMPTE, 2024, ISBN: [object Object]. Available at <a href="https://doi.org/10.5594/MOO/3043" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3043</a></span>

SMPTE's HTML Pub

Preview:

Denys Krasnikov and Alexander Konovalov; AI Translation and Dubbing with Emotional Accuracy, MTS 2024, Article 32 (pp. 1 to 16); SMPTE, 2024, ISBN: [object Object]
doi: 10.5594/MOO/3043
url: https://doi.org/10.5594/MOO/3043

Snippet:

<li>
Denys Krasnikov and Alexander Konovalov; <cite id="bib-10-5594-moo-3043">AI Translation and Dubbing with Emotional Accuracy</cite>, MTS 2024, Article 32 (pp. 1 to 16); SMPTE, 2024, ISBN: [object Object]
<span class="doi">10.5594/MOO/3043</span>
</li>