Architecture for Embedding Audiovisual Feature Extraction Tools in Archives

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 120, No. 1, pp. 29–36
Abstract: Soon, it will no longer be sufficient for only archivists to annotate audiovisual material. Not only is the number of archivists limited, but the time they spend on annotating one item is insufficient to create time-based and detailed descriptions about the content to make fully optimized video search possible. Furthermore, as a result of file-based production methods, we observe an accelerated increase in newly created audiovisual material that must be described. Fortunately, high-quality feature extraction (FE) tools are increasingly being developed by research institutes. These tools examine the audiovisual essence and return particular information about the analyzed video, audio, or both streams. For example, the tools can automatically detect shot boundaries, detect and recognize faces and objects, and segment audio streams. As a result, they quickly and cheaply generate metadata that can be used for indexing and searching. In addition, they relieve archivists of the need to perform tedious, repetitive, but necessary low-added value tasks, such as identifying within an audio stream the speech and the music segments. Although most tools are not yet commercially offered, these solutions are expected to become available soon for broadcasters and media companies alike. This paper describes a solution for integrating such FE tools within the annotation workflow of a media company. This solution, in the form of an architecture and workflow, is scalable, extensible, and loosely coupled and has clear and easy-to-implement interfaces. As such, our architecture allows additional tools to be plugged in irrespective of the software and hardware used by the media company. By integrating FE tools within the workflow of the annotating audiovisual essence, more and better metadata can be created, allowing other tools to improve indexing, search, and retrieval of media material within audiovisual archives.
Publication Date: 2011-01-01
DOI: 10.5594/j18005XY
ISSN: Print: 1545-0279
Link: https://doi.org/10.5594/j18005XY
Author(s): Robbie De Sutter
Karel Braeckman
Copyright: © 2011 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

1 Hoffman J. Soetens P. De Geyter M. “An Integrated File-Based Media Production Workflow: The Digital Media Factory,” Proc. IBC ‘07 Amsterdam, the Netherlands Sept. 2007 . EXTERNAL
10 Autonomy virage, http://www.virage.com/ . EXTERNAL
11 Cluster Media Labs, http://www.clustermedialabs.com/ . EXTERNAL
12 Pictron, http://www.pictron.com/ . EXTERNAL
13 Fraunhofer Mood-player, http://www.fraunhofer.de/en/ . EXTERNAL
14 iPharro, http://www.ipharro.com/ . EXTERNAL
15 ina, http://www.ina-entreprise.com/ . EXTERNAL
16 EBU “Content Analysis-Based Automatic Information Extraction,” http://tech.ebu.ch/groups/pscaie . EXTERNAL
17 NIST “Digital Video Retrieval,” www-nlpir.nist.gov/projects/trecvid/ ; www.ibbt.be/en/project/pisa . EXTERNAL
18 IBBT “Production, Indexing and Search of Audio–Visual Material,” www. ibbt.be/en/project/pisa . EXTERNAL
19 Braeckman K. De Sutter R. Verwaest M. “Building Custom Video Search Upon an Existing MAM System,” 5th IASTED European Conf. Internet and Multimedia Systems and Applications (EuroIMSA ) Cambridge, UK July 2009 . EXTERNAL
2 De Bruyne S. De Neve W. De Wolf K. De Schrijver D. Verhoeve P. Van de Walle R. “Temporal Video Segmentation on H.264/AVC Compressed Bitstreams,” LNCS 4351 2007 , 1 12. EXTERNAL
20 De Geyter M. Oorts N. Overmeire L. “Integration Demands on MAM Systems: A Proof of Concept Solution,” SMPTE J 117 38 46 November/December 2008 . EXTERNAL
21 Van Assche S. Van der Weken D. Muylaert B. Desmet S. Volckaert B. “Automating Workflows in Media Production,” Proc. 3rd Int. Conf. ENASE ‘08 pp. 31 38, May 2008 . EXTERNAL
22 Van der Weken D. Desmet S. Muylaert B. Van Assche S. Volckaert B. “Automating Media Processes in a Service Oriented Architecture,” Proc. ISCC ‘08 pp. 97 102, July 2008 . EXTERNAL
23 IPTC, “NewsML-G2 Specification ver. 2.0 rev. 2,” www.iptc.org/ , January 2008. EXTERNAL
24 EBU, “P/Meta Semantic Metadata Schema,” Technical Report 3295, ver. 2.0, http://tech.ebu.ch/docs/tech/tech3295v2.pdf , July 2007 . EXTERNAL
25 Dublin Core, “Metadata Initiative,” ver. 1.1, http://dublincore.org , January 2008. EXTERNAL
26 EBU, “Core Metadata Set,” Technical Report 3293-2008 , ver. 1.0, http://tech.ebu.ch/lang/en/MetadataEbuCore , December 2008. EXTERNAL
27 ISO/IEC JTC1/SC29/WG11, “MPEG-7,” http://www.chiariglione.org/mpeg/standards/mpeg-7/mpeg-7.htm , October 2004. EXTERNAL
3 Osian M. Van Gool L. “Video Shot Characterization,” Mach. Vision App ., 15 ( 3) 17 177, 2004 . EXTERNAL
4 Yusoff Y. Christmas W. J. Kittler J. V. “Video Shot Cut Detection Using Adaptive Thresholding,” Proc. 11th British Mach. Video Conf 2000 . EXTERNAL
5 Zhao W. Chellappa R. Phillips P. J. Rosenfeld A. “Face Recognition: A Literature Survey,” ACM Comput. Surv 35 ( 4) 399 458, 2003 . EXTERNAL
6 De Smet M. Fransens R. Van Gool L. “A Generalized EM Approach for 3D Model Based Face Recognition Under Occlusions,” Proc. CVPR Vol. 2 pp. 1423 1430, 2006 . EXTERNAL
7 Patsis Y. Verhelst W. “A Speech/Music/Silence/Garbage Classifier for Searching and Indexing Broadcast News Material,” Proc. 2008 19th Int. Conf. Database and Expert Systems Application Turin, Italy pp. 585 589, 1-5 Sept. 2008. EXTERNAL
8 Abu-El-Quran A. R. Chan A. D. C. Goubran R. A. “Adaptive Feature Selection for Speech/Music Classification,” IEEE 8th Workshop on Multimedia Signal Processing pp. 212 216, 2006 . EXTERNAL
9 Memnon, http://www.memnon.be/ . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-j18005XY.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Robbie De Sutter and Karel Braeckman; Architecture for Embedding Audiovisual Feature Extraction Tools in Archives, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 1, 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18005XY

Snippet:

Robbie De Sutter and Karel Braeckman; Architecture for Embedding Audiovisual Feature Extraction Tools in Archives, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 1, 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18005XY

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Robbie De Sutter and Karel Braeckman; <cite>Architecture for Embedding Audiovisual Feature Extraction Tools in Archives</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 1, 2011); SMPTE, 2011. Available at <a href="https://doi.org/10.5594/j18005XY" target="_blank" rel="noopener">https://doi.org/10.5594/j18005XY</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Robbie De Sutter and Karel Braeckman; <cite id="bib-10-5594-j18005xy">Architecture for Embedding Audiovisual Feature Extraction Tools in Archives</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 1, 2011); SMPTE, 2011
<span class="doi">10.5594/j18005XY</span>
</li>