API Build-data JSON Resources
Theme

Choose how MSRBot.io looks on this device.

Auto (match system) Light mode Dark mode

Preference is stored in this browser only.

SMPTE Meetings and Conferences ( October 2017)
[ACTIVE]

Exploring Image Corruption in the Workflow, and how to Stop this from Happening

Metadata

Publisher
SMPTE — White Plains, NY, USA
Doc Type
Conference Paper
Content Type
Original Research
Volume
2017, No. 10, pp. 1–21
Abstract
LTO tape Systems, SAN, NAS, Object Store, RAM, and WAN Optimizers all have configurations available to protect the fidelity of image contents in the workflow. Each system has different methods to ensure that frames contents aren't corrupted, and each method protects against a variety of different failure conditions. And yet, unrecoverable frame corruption still occurs at an unacceptable level. This problem is even more serious in an archive scenario, where content may sit untouched for a long duration, and with the corruption staying undetected for extended periods of time. Media and Entertainment workflows rely solely on the protections provided by the underlying compute, storage, and transmission technologies to ensure the fidelity of the data in the workflow. These protections are very difficult to fully characterize and track, because the technologies are generally deployed deep in the hardware or at the lowest software levels of the IT infrastructure. Depending on the path a frame takes through the workflow, it will be treated to a varying set of protection technologies, like RAID, erasure coding, ECC Memory, and parity checking. To overcome the uncertainty of associated with how these methods ensure fidelity, the industry employs failure detection at each stage of the workflow (generally MD5 checksums). When a corrupt image is detected, the entire file must be restored from a backup copy. This approach is very expensive and introduces significant delays in the production when errors occur. This paper will explore the IT technology utilized in the workflow, and discuss the protection mechanisms provided or employed by each workflow element. In this context, the paper will discuss how frame corruption can occur, even when all of the protection technologies are working as designed. The paper will also discuss a method for providing protection to images at the frame level using Forward Error Correction, such that there is uniformity of protection for images applied throughout the workflow, and such that media errors may be recovered in most cases without having to access a backup copy.
Publication Date
2017-10-23
DOI
10.5594/M001797
ISBN
Print: 978-1-61482-959-1
Link
https://doi.org/10.5594/M001797
Author(s)
Keith R. HoganPixspan, Inc.
Keyword(s)
Data Corruption, Forward Error Correction, Storage, Archive, SSDs, Object Storage, SAN, NAS, ECC, RAID, Erasure Coding, Raster, Metadata, MD5, Hash Function
Copyright
© 2017 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001797.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree
Reference this Doc

Plain text (ISO 690 compliant)

Preview:
Keith R. Hogan; Exploring Image Corruption in the Workflow, and how to Stop this from Happening, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at https://doi.org/10.5594/M001797
Snippet:
Keith R. Hogan; Exploring Image Corruption in the Workflow, and how to Stop this from Happening, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at https://doi.org/10.5594/M001797

HTML (ISO 690 compliant)

Preview:
Keith R. Hogan; Exploring Image Corruption in the Workflow, and how to Stop this from Happening, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at https://doi.org/10.5594/M001797
Snippet:
<span class="citation">Keith R. Hogan; <cite>Exploring Image Corruption in the Workflow, and how to Stop this from Happening</cite>, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1. Available at <a href="https://doi.org/10.5594/M001797" target="_blank" rel="noopener">https://doi.org/10.5594/M001797</a></span>

SMPTE Icon SMPTE's HTML Pub

Preview:
Keith R. Hogan; Exploring Image Corruption in the Workflow, and how to Stop this from Happening, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1
doi: 10.5594/M001797
url: https://doi.org/10.5594/M001797
Snippet:
<li>
Keith R. Hogan; <cite id="bib-10-5594-m001797">Exploring Image Corruption in the Workflow, and how to Stop this from Happening</cite>, SMPTE Meetings and Conferences ( October 2017); SMPTE, 2017, ISBN: 978-1-61482-959-1
<span class="doi">10.5594/M001797</span>
</li>