Video Production with Generative AI

Metadata

Publisher: SMPTE — Hollywood, CA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 00, pp. 1–10
Abstract: The number of applications for Generative AI (GenAI) in video production is rapidly increasing. Many kinds of GenAI models are used, from audio generation to text generation models. However, video generation models (VGMs) are foundational for video production and are gaining importance as their power and expressiveness increases. In regard to expressiveness, VGMs are different from prior kinds of GenAI models in that VGM users must have domain-specific knowledge in order to optimally leverage VGMs to express their artistic vision. More specifically, it can be demonstrated that VGM users must know at least the basics of the visual language of cinematography in order to properly translate their artistic vision into quality VMG output. The tool by which users apply cinematographic language to VGMs is prompt engineering, which is used to craft VGM output into something useful and aesthetically viable. Using prompt engineering, it is possible to ask a VGM to instantiate the basic elements of cinematography such as: camera placement, camera movement, shot composition, shot size, focus/depth of field, and lighting. Additionally, GenAI can be used to create storyboards and animatics to help plan out these cinematographic elements. VGMs and other GenAI models also are useful in post-production workflows. This includes not only editing, but also other post-production tasks such as visual effects.
Publication Date: 2024-10-21
DOI: 10.5594/MOO/3041
ISBN: [object Object]
Link: https://doi.org/10.5594/MOO/3041
Author(s): Brent Rabowsky
Keyword(s): Generative AI, GenAI, Video Generation, Video Production
Copyright: © 2024 SMPTE

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-MOO-3041.json

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Brent Rabowsky; Video Production with Generative AI, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3041

Snippet:

Brent Rabowsky; Video Production with Generative AI, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3041

HTML (ISO 690 compliant)

Preview:

Brent Rabowsky; Video Production with Generative AI, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]. Available at https://doi.org/10.5594/MOO/3041

Snippet:

<span class="citation">Brent Rabowsky; <cite>Video Production with Generative AI</cite>, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]. Available at <a href="https://doi.org/10.5594/MOO/3041" target="_blank" rel="noopener">https://doi.org/10.5594/MOO/3041</a></span>

SMPTE's HTML Pub

Preview:

Brent Rabowsky; Video Production with Generative AI, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]
doi: 10.5594/MOO/3041
url: https://doi.org/10.5594/MOO/3041

Snippet:

<li>
Brent Rabowsky; <cite id="bib-10-5594-moo-3041">Video Production with Generative AI</cite>, MTS 2024, Article 30 (pp. 1 to 10); SMPTE, 2024, ISBN: [object Object]
<span class="doi">10.5594/MOO/3041</span>
</li>