Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprints

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 119, No. 3, pp. 42–52
Abstract: A method for measuring and maintaining time synchronization between an A/V stream (A/V) is described. A/V fingerprints are used to create a combined A/V synchronization signature (A/V Sync Signature) at a known reference point. This signature is used at later points to measure A/V timing synchronization relative to the reference point. This method may be used, for example, to automatically detect and correct A/V synchronization (i.e., lip-sync) errors in broadcast systems and other applications. Advantages of the method described over other existing methods include that it does not require modification of the audio or video signals, it can respond to dynamically changing synchronization errors, and it is designed to be robust to modifications of the A/V signals. Although the system requires data to be conveyed to the detection point, this data does not need to be synchronized with, or directly attached to, the audio or video streams. This method uses fingerprints; it also enables other fingerprinting applications within systems, such as content identification and verification. In addition, it may be used to maintain synchronization of other metadata associated with A/V streams.
Publication Date: 2010-04-01
DOI: 10.5594/J11398
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/J11398
Author(s): Kent Terry
Regunathan Radhakrishnan
Copyright: © 2010 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

1. Radhakrishnan R. Bauer C. Cheng C. Terry K. , “Audio Signature Extraction Based on Projections of Spectrograms,” Proc. IEEE Interntional Conference on Multimedia and Expo, Beijing, China, 2007 . EXTERNAL
10. Fridrich J. Goljan M. , “Robust Hash Functions for Digital Watermarking,” Proc. of International Symposium on Information Technology, Las Vegas, 2000 . EXTERNAL
2. Radhakrishnan R. Bauer C. , “Content-based Video Signatures Based on Projections of Difference Images,” IEEE Workshop on Multimedia Signal Processing, Crete, Greece, 2007 . EXTERNAL
3. Radhakrishnan R. Terry K. Bauer C. , “Audio and Video Signatures for Synchronization,” Proc. IEEE International Conf. on Multimedia and Expo, Hannover, Germany, 2008 . EXTERNAL
4. Kozat S. S. Venkatesan R. Mihcak M. K. , “Robust Perceptual Image Hashing via Matrix Invariants,” IEEE International Conference on Image Processing, Singapore, 2004 . EXTERNAL
5. Swaminathan A. Mao Y. Wu Min , “Image Hashing Resilient To Geometric and Filtering Operations,” IEEE Workshop on Multimedia Signal Processing, Siena, Italy, 2004 . EXTERNAL
6. Burges C.J.C. Platt J. C. Jana S. , “Distortion Discriminant Analysis for Audio Fingerprinting,” IEEE Transactions on Speech and Audio Processing, May 2003 . EXTERNAL
7. Haitsma J. Kalker T. , “A Highly Robust Audio Fingerprinting System,” Proc. International Conference on Music Information Retrieval, Paris, France, 2002 . EXTERNAL
8. Lu Chun-Shien , “Audio Fingerprinting Based on Analyzing Time-frequency Localization of Signals,” IEEE Workshop on Multimedia Signal Processing, St. Thomas, Virgin Islands, 2002 . EXTERNAL
9. Batlle E. Masip J. Guaus E. , “Automatic Song Identification in Noisy Broadcast Audio,” Proc. Signal and Image Processing , Aug. 2002 . EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-J11398.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Kent Terry and Regunathan Radhakrishnan; Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprints, SMPTE Motion Imaging Journal ( Volume: 119, Issue: 3, April 2010); SMPTE, 2010. Available at https://doi.org/10.5594/J11398

Snippet:

Kent Terry and Regunathan Radhakrishnan; Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprints, SMPTE Motion Imaging Journal ( Volume: 119, Issue: 3, April 2010); SMPTE, 2010. Available at https://doi.org/10.5594/J11398

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Kent Terry and Regunathan Radhakrishnan; <cite>Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprints</cite>, SMPTE Motion Imaging Journal ( Volume: 119, Issue: 3, April 2010); SMPTE, 2010. Available at <a href="https://doi.org/10.5594/J11398" target="_blank" rel="noopener">https://doi.org/10.5594/J11398</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Kent Terry and Regunathan Radhakrishnan; <cite id="bib-10-5594-j11398">Detection and Correction of Lip-Sync Errors Using Audio and Video Fingerprints</cite>, SMPTE Motion Imaging Journal ( Volume: 119, Issue: 3, April 2010); SMPTE, 2010
<span class="doi">10.5594/J11398</span>
</li>