Interoperable AV Sync Systems in the SMPTE 22TV Lip Sync AHG: Content-Fingerprinting-Based Audio-Video Synchronization

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 120, No. 5, pp. 47–57
Abstract: An end-to-end interoperable content fingerprint-based audio-video synchronization system is described, including an audio-video fingerprinting technology overview and current status of the standardization process in the SMPTE 22TV Lip Sync Ad Hoc Group (AHG). The system is based on the audio-video fingerprinting technology currently submitted to the SMPTE 22TV Lip Sync AHG by Miranda Corp. The particular type of signatures are implemented and embedded in Zeitera Corp.'s file-based workflow, and in Miranda's realtime SDI-based audio-video synchronization equipment, showing signatures being captured by either vendor's system and providing results through either vendor's application management flow. An example of successful collaboration on an interoperable, mixed-vendor solution that can handle both compressed audio-video files and realtime SDI audio-video input is highlighted.
Publication Date: 2011-07-01
DOI: 10.5594/j18060XY
ISSN: Print: 1545-0279
Link: https://doi.org/10.5594/j18060XY
Author(s): Mihailo Stojancic
bio
Mihailo Stojancic has more than 25 years of experience in the design and implementation of digital video processing systems, and complex VLSI architectures for high-performance computing systems. Currently, he is the CTO at Zeitera Corp., where he is responsible for audio/video fingerprinting technology development. He has held numerous leading technical and managerial positions in Cisco, IBM, NEC, and multiple startup companies. Earlier in his career, as a senior staff member with the IBM, Inc., Stojancic made pioneering contributions to the development of IBM's MPEG-2 video encoder/decoder components for digital set-top-box applications. Stojancic has contributed to professional journals and conference proceedings and has filed 24 US patents/patent applications. He was an active member of the SMPTE C24-Digital Video standards committee and currently he is an active contributor to the SMPTE's 22TV Lip Sync AHG working on content-fingerprint based audio/video synchronization. He holds a Ph.D. degree in electrical engineering from the City University of New York, and MS/BS degrees in electrical and computer engineering from the University of Belgrade.
Daniel Eakins
bio
Dan Eakins is a Silicon Valley entrepreneur who has been leading technology firms since 1989. He has specialized in networking, security, processor design, and video compression. He is currently CEO of Zeitera and engaged in developing markets for Zeitera's video and audio fingerprinting technology, with special focus on linear broadcast television.
Copyright: © 2011 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

1 Li Chengliang “Acoustic Speech to Lip Feature Mapping for Multimedia Applications,” Proc. 3rd Int. Symp. on Image and Signal Proc. and Analys . Sept. 2003 , Rome, Italy. EXTERNAL
2 Voyatzis G. “The Use of Watermarks in the Protection of Digital Multimedia Products,” Proc. of the IEEE 87 ( 7) 1197 1202, July 1999 . EXTERNAL
3 Cheung W. N. “Digital Image Watermarking In Spatial and Transform Domains,” Proc. TENCON 2000 3 374 378, Sept. 2000. EXTERNAL
4 Chandra M. “A DWT Domain Visible Watermarking Techniques for Digital Images,” 2010 Int.Conf. on Elect. and Inform. Engineer Kyoto, Japan. EXTERNAL
5 Megalingam R. K. “Performance Comparison of Novel, Robust Spatial Domain Digital Image Watermarking with the Conventional Frequency Domain Watermarking,” 2010 Int.Conf. on Signal Acquis. and Proc . Bangalore, India. EXTERNAL
6 Sharma R. K. Decker S. “Practical Challenges for Digital Watermarking Applications,” Digimark Corp 2001 . EXTERNAL
7 Terry K. Radhakrishnan R. “Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprint,” SMPTE Mot. Imag.J . 119 ( 3) 42 52, April 2010 . EXTERNAL
8 Radhakrishnan R. Terry K. Bauer C. “Audio and Video Signatures for Synchronization,” 2008 IEEE Int.l Conf. on Multi. & Expo June 2008 , Hannover, Germany pp. 1549 1552. EXTERNAL
9 Stojancic M. “Methods and Apparatus for Providing a Scalable Identification of Digital Video Sequences,” Patent Application June 18, 2008 . EXTERNAL
10 Stojancic M. “Digital Video Fingerprinting Based on Resultant Weighted Gradient Orientation Computation,” Patent Application June 25, 2009 . EXTERNAL
11 Radhakrishnan R. Bauer C. “Content-Based Video Signatures Based on Projections of Diff erence Images,” IEEE 9th Workshop on Multimedia Signal Processing, 2007, (MMSP 2007 ) pp. 341 344. EXTERNAL
12 Bober M. Brasnett P. “MPEG-7 Visual Signature Tools,” IEEE Int.Conf. on Multi. and Expo (ICME 2009 ) pp. 1540 1543. EXTERNAL
13 Brasnett P. Paschalakis S. Bober M. “Recent Developments on Standardization of MPEG-7 Visual Signature Tools,” 2010 IEEE Int. Conf. on Multi. and Expo (ICME 2010 pp. 1347 1352. EXTERNAL
14 Chien S.-Y. “Fast Video Segmentation Algorithm with Shadow Cancellation, Global Motion Compensation, and Adaptive Threshold Techniques,” IEEE Trans. on Multi . 6 ( 5) 732 748, Oct. 2004 EXTERNAL
15 Seo J. S. Haitsma J. Kalker T. Yoo C. D. “A Robust Image Fingerprinting System Using the Radon Transform,” Sig. Proc. Image Comm . 19 325 339, 2004 . EXTERNAL
16 Lowe David G. “Distinctive Image Features from Scale-Invariant Key-points,” Int. J. Comput. Vis . 60 ( 2) Nov. 2004 . EXTERNAL
17 Mikolajczyk K. Schmid C. “Scale & Affine Invariant Interest Point Detectors,” Int. J. Comput. Vis . 60 ( 1) 63 86, 2004 . EXTERNAL
18 Grabner M. Grabner H. Bischof H. “Fast Approximated SIFT,” Institute for Computer Graphics and Vision, Graz University of Technology Graz, Austria Jan. 2004 . EXTERNAL
19 Bay H. Tuytelaars T. Van Gool L. “SURF: Speeded Up Robust Features,” ETH Zurich 2006 . EXTERNAL
20 Stojancic M. “Digital Video Content Fingerprinting Based on Scale Invariant Interest Region Detection with an Array of Anisotropic Filters,” Patent Application, Nov. 5, 2009 . EXTERNAL
21 Burges C. J. C. “Extracting Noise-Robust Features from Audio Data,” Proc. IEEE Int. Conf. on Acoust., Speech and Sig. Proc . 2002 , Orlando, Florida pp. I-1021 I-1024. EXTERNAL
22 Radhakrishnan R. “Audio Signature Extraction Based on Projections of Spectrograms,” IEEE Int. Conf. on Multi. & Expo 2007 , Beijing, China pp. 2110 2113. EXTERNAL
23 Cano P. “A Review of Algorithms for Audio Fingerprinting,” IEEE Workshop on Multi. Signal Process . 2002 , St. Thomas, Virgin Islands, USA. EXTERNAL
24 Haitsma J. Kalker T. “A Highly Robust Audio Fingerprinting System,” Proc.3rd Int. Conf. on Music Inform. Retrieval (ISMIR 2002 ) 2002 , Paris, France. EXTERNAL
25 Baluja S. Covell M. “Content Fingerprinting Using Wavelets,” Proc.,3rd European Conf.on Vis.Media Production (CVMP ) 2006 . EXTERNAL
26 Baluja S. Covell M. “Audio Fingerprinting: Combining Computer Vision & Data Stream Processing,” Google 2007 . EXTERNAL
27 Ke Y. “Computer Vision for Music Identification,” Proc. IEEE Comp.Society Conf. on Comp.Vis.and Pattern Recog . 2005 , vol. 1 pp. 597 604. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-j18060XY.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Mihailo Stojancic and Daniel Eakins; Interoperable AV Sync Systems in the SMPTE 22TV Lip Sync AHG: Content-Fingerprinting-Based Audio-Video Synchronization, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 5, July 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18060XY

Snippet:

Mihailo Stojancic and Daniel Eakins; Interoperable AV Sync Systems in the SMPTE 22TV Lip Sync AHG: Content-Fingerprinting-Based Audio-Video Synchronization, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 5, July 2011); SMPTE, 2011. Available at https://doi.org/10.5594/j18060XY

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Mihailo Stojancic and Daniel Eakins; <cite>Interoperable AV Sync Systems in the SMPTE 22TV Lip Sync AHG: Content-Fingerprinting-Based Audio-Video Synchronization</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 5, July 2011); SMPTE, 2011. Available at <a href="https://doi.org/10.5594/j18060XY" target="_blank" rel="noopener">https://doi.org/10.5594/j18060XY</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Mihailo Stojancic and Daniel Eakins; <cite id="bib-10-5594-j18060xy">Interoperable AV Sync Systems in the SMPTE 22TV Lip Sync AHG: Content-Fingerprinting-Based Audio-Video Synchronization</cite>, SMPTE Motion Imaging Journal ( Volume: 120, Issue: 5, July 2011); SMPTE, 2011
<span class="doi">10.5594/j18060XY</span>
</li>