A Fully AI Approach to Descriptive Video Accessibility

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–13
Abstract: A highly manual and time-consuming process has traditionally created descriptive video accessibility tracks, but it is now possible to apply automated software to produce high-quality and nuanced descriptive services. We break down this process including steps for transcription, generative AI scripting using large language models (LLM), generation of synthetic voice, and mixing, and show examples for several different program types. We also demonstrate a heuristic model for descriptive video quality scoring and its application to the AI-generated descriptions created using our methods. Finally, we assess dimensions of quality descriptive video where AI continues to struggle and further technology improvements may be required to match the utility of traditional workflows.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3044
ISBN: Electronic: 978-1-61482-965-2
Link: https://doi.org/10.5594/MOO/3044
Author(s): Bill McLaughlin
James Hu
Fahad Ahmad Arsal
Rhys Fuller
Keyword(s): Descriptive Video, Audio Description, Accessibility, Generative AI, Large Language Models (LLMs)
Copyright: © 2024 SMPTE

Bibliographic Reference(s)

EBU, “Technical Review - Audio Description Dip Levels for Non-Sound Engineers”, forthcoming publication from EBU Media Access Technology Group. EXTERNAL
“Tears of Steel”, Directed by Ian Hubert. Blender Foundation, 2012. EXTERNAL
“Angel”, 1995. “Law & Order” Season 6, Episode 8. Wolf Entertainment / Universal Television. EXTERNAL
“Wing It!”, Directed by Rick Schutte. Blender Studios, 2023. EXTERNAL
“Out Of Here” (Episode 1), 2024. Blackwood River Films. EXTERNAL
Pablo Romero-Fresco: Subtitling through Speech Recognition: Respeaking. Manchester: St. Jerome 2011. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3044.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Bill McLaughlin, James Hu, Fahad Ahmad Arsal, and Rhys Fuller; A Fully AI Approach to Descriptive Video Accessibility, MTS 2024, Article 33 (pp. 1 to 13); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3044

Snippet:

Bill McLaughlin, James Hu, Fahad Ahmad Arsal, and Rhys Fuller; A Fully AI Approach to Descriptive Video Accessibility, MTS 2024, Article 33 (pp. 1 to 13); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at https://doi.org/10.5594/MOO/3044

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Bill McLaughlin, James Hu, Fahad Ahmad Arsal, and Rhys Fuller; <cite>A Fully AI Approach to Descriptive Video Accessibility</cite>, MTS 2024, Article 33 (pp. 1 to 13); SMPTE, 2024, ISBN: 978-1-61482-965-2. Available at <a href="https://doi.org/10.5594/MOO/3044" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3044</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Bill McLaughlin, James Hu, Fahad Ahmad Arsal, and Rhys Fuller; <cite id="bib-10-5594-moo-3044">A Fully AI Approach to Descriptive Video Accessibility</cite>, MTS 2024, Article 33 (pp. 1 to 13); SMPTE, 2024, ISBN: 978-1-61482-965-2
<span class="doi">10.5594/MOO/3044</span>
</li>