Architecture for Embedding Audiovisual Feature Extraction Tools in Archives

Metadata

Publisher: SMPTE — White Plains, NY
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2009, No. 10, pp. 1–10
Abstract: In the near future, it will no longer be sufficient that only archivists annotate audiovisual material. Not only is the number of archivists limited, the time they can spend on annotating one item is insufficient to create time-based and detailed descriptions about the content to make fully optimized video search possible. Furthermore, we observe an accelerated increase in newly created audiovisual material that must be described due to introduction of file-based production methods. Fortunately, more and more high-quality feature extraction tools are being developed by research institutes. These tools examine the audiovisual essence and return particular information about the analyzed video and/or audio streams. For example, tools can automatically detect shot boundaries, detect and recognize faces and objects, segment audio streams, etc. As a result, they quickly and cheaply generate metadata that can be used for indexing and searching. On top of that, it relieves archivists of performing tedious and repetitive, but necessary low-added value tasks, for example identifying within an audio stream the speech and music segments. Although most tools are currently not yet commercially offered, it is to be expected that these solutions soon will become available for broadcasters and media companies alike. In this paper, we describe a solution on how to integrate such feature extraction tools within the annotation workflow of a media company. This solution, in the form of an architecture and workflow, is scalable, extensible, loosely coupled, and has clear and easy to implement interfaces. As such, our architecture allows one to plug in additional tools irrespective of the software and hardware used by the media company. By integrating feature extraction tools within the workflow of annotating audiovisual essence, more and better metadata can be created allowing other tools to improve indexing, search and retrieval of media material within audiovisual archives.
Publication Date: 2009-10-01
DOI: 10.5594/M001332
Link: https://doi.org/10.5594/M001332
Author(s): Robbie De SutterVRT-medialab, A. Reyerslaan 52, B-1043 Brussel, Belgium
Karel BraeckmanVRT-medialab, A. Reyerslaan 52, B-1043 Brussel, Belgium
Keyword(s): Media Asset Management, Architecture
Copyright: © 2009 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

[1] Hoffman J. Soetens P. De Geyter M. , An integrated file-based media production workflow: the digital media factory . Proc. of IBC 2007, Amsterdam (The Netherlands), September 2007 . EXTERNAL
[10] NIST Digital Video Retrieval , http://www-nlpir.nist.gov/projects/trecvid/ . EXTERNAL
[11] IBBT Production, Indexing and Search of Audio-visual material , http://www.ibbt.be/en/project/pisa . EXTERNAL
[12] Braeckman K. De Sutter R. Verwaest M. , Building custom video search upon an existing MAM system , The Fifth IASTED European Conference on Internet and Multimedia Systems and Applications (EuroIMSA), Cambridge (UK), July 2009 . EXTERNAL
[13] De Geyter M. Oorts N. Overmeire L. , Integration Demands on MAM Systems: A Proof of Concept Solution , SMPTE Journal , 2008 November / December. EXTERNAL
[14] Van Assche S. Van der Weken D. Muylaert B. Desmet S. Volckaert B. , Automating Workflows in Media Production, in Proceedings of the 3rd International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE'2008), May 2008 , 31 – 38 . EXTERNAL
[15] Van der Weken D. Desmet S. Muylaert B. Van Assche S. Volckaert B. , Automating Media Processes in a Service Oriented Architecture , ISCC'2008, July 2008 , 97–102. EXTERNAL
[16] International Press Telecommunications Council (IPTC), NewsML-G2 specification version 2.0 rev. 2, http://www.iptc.org/ , January 2008 . EXTERNAL
[17] EBU P/Meta Semantic Metadata Schema., technical report 3295 v2.0, http://tech.ebu.ch/docs/tech/tech3295v2.pdf , July 2007. EXTERNAL
[18] Dublin Core Metadata Initiative, http://dublincore.org , v. 1.1, January 2008 . EXTERNAL
[19] EBU Core Metadata Set, technical report 3293–2008, http://tech.ebu.ch/lang/en/MetadataEbuCore , version 1.0, December 2008 . EXTERNAL
[2] De Bruyne S. De Neve W. De Wolf K. De Schrijver D. Verhoeve P. Van de Walle R. , Temporal video segmentation on H.264/AVC compressed bitstreams , LNCS 4351, 2007 , 1 – 12 . EXTERNAL
[20] ISO/IEC JTC1/SC29/WG11, MPEG-7, http://www.chiariglione.org/mpeg/standards/mpeg-7/mpeg-7.htm , October 2004 . EXTERNAL
[3] Osian M. Van Gool L. , Video shot characterization , Machine Vision and Applications , vol. 15 ( 3 ), 2004 , 172 – 177 . EXTERNAL
[4] Yusoff Y. Christmas WJ. Kittler JV. , Video Shot Cut Detection Using adaptive thresholding . Proceedings of the 11th British Machine Video Conference, 2000 . EXTERNAL
[5] Zhao W. Chellappa R. Phillips P. J. Rosenfeld A. , Face recognition: a literature survey . ACM Comput. Surv. , Vol. 35 ( 4 ), 2003 , 399 – 458 . EXTERNAL
[6] De Smet M. Fransens R. Van Gool L. , A generalized EM approach for 3D model based face recognition under occlusions . Proc. CVPR , Vol 2 , 2006 , 1423 – 1430 . EXTERNAL
[7] Patsis Y. Verhelst W. , A Speech / Music / Silence / Garbage classifier for searching and indexing broadcast news material , Proceedings of the 2008 19th International Conference on Database and Expert Systems Application, Turin (Italy), 1–5 September 2008 , 585–589. EXTERNAL
[8] Abu-El-Quran A.R. Chan A.D.C. Goubran R.A. , Adaptive Feature Selection for Speech / Music Classification , IEEE 8th Workshop on Multimedia Signal Processing, 2006 , 212 – 216 . EXTERNAL
[9] EBU Content Analysis-based Automatic Information Extraction , http://tech.ebu.ch/groups/pscaie . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001332.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Robbie De Sutter and Karel Braeckman; Architecture for Embedding Audiovisual Feature Extraction Tools in Archives, SMPTE Meetings and Conferences ( October 2009); SMPTE, 2009. Available at https://doi.org/10.5594/M001332

Snippet:

Robbie De Sutter and Karel Braeckman; Architecture for Embedding Audiovisual Feature Extraction Tools in Archives, SMPTE Meetings and Conferences ( October 2009); SMPTE, 2009. Available at https://doi.org/10.5594/M001332

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Robbie De Sutter and Karel Braeckman; <cite>Architecture for Embedding Audiovisual Feature Extraction Tools in Archives</cite>, SMPTE Meetings and Conferences ( October 2009); SMPTE, 2009. Available at <a href="https://doi.org/10.5594/M001332" target="_blank" rel="noopener">https://doi.org/10.5594/M001332</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Robbie De Sutter and Karel Braeckman; <cite id="bib-10-5594-m001332">Architecture for Embedding Audiovisual Feature Extraction Tools in Archives</cite>, SMPTE Meetings and Conferences ( October 2009); SMPTE, 2009
<span class="doi">10.5594/M001332</span>
</li>