Implementing Hyperscale and HPC Techniques and Technologies Necessary to Modern Media and Broadcast Workflows

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2018, No. 10, pp. 1–6
Abstract: Across the media production and broadcast industries, studios of every size are accelerating their consumption of vast amounts of storage resources. With increasing quantities of data produced from increasingly higher-resolution digital cameras and content featuring greater color depth across a broader color spectrum, file manipulation and storage capacity have now reached the scale and performance requirements of high-performance computing (HPC) applications. Broadcast, production, post-production and distribution now generate hyperscale workloads with tens to hundreds of petabytes of business-critical data and streams of several gigabytes/second (GB/s) per client. For media companies to remain competitive, they need to apply the lessons learned by hyperscaler and HPC companies to cost-effectively and consistently provide high levels of performance with a very small team while ensuring long-term data protection. — This paper and presentation will utilize customer and industry examples from hyperscaler, HPC, and Media and Entertainment (M&E) organizations to discuss in tutorial fashion and learn how to implement a hyperscaler-like storage infrastructure that provides limitless scalable storage to all applications across a broad range of workloads; utilize commodity servers without hardware redundancy, RAID controllers or NVRAM required for the lowest cost and flexibility; and remove the challenges of deploying and managing infrastructure, enabling new methods of creating and delivering content.
Publication Date: 2018-10-22
DOI: 10.5594/M001852
ISBN: Print: 978-1-61482-960-7
Link: https://doi.org/10.5594/M001852
Author(s): Bjorn KolbeckCo-Founder and CEO, Quobtye® Inc., 4633 Old Ironsides Drive #150, Santa Clara, CA 95054, USA
Keyword(s): Storage, High-performance Computing (HPC), Hyperscaler, Throughput, Capacity
Copyright: © 2018 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

Quobyte Inc, "MX1 Case Study", Available: https://www.quobyte.com/case-studies/mx1 EXTERNAL
L. Lamport, “Paxos Made Simple,” November 2001 EXTERNAL
L. A. Barroso, J. Clidaras, U. Holzle, “The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Second Edition,” 2013 EXTERNAL
Mike Burrows, "The Chubby lock service for loosely-coupled distributed systems" in 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2006 EXTERNAL
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, "The Google file system" in SOSP ’03: Proceedings of the nineteenth ACM symposium on Operating systems principles, pages 29–43, New York, NY, USA, 2003. ACM Press. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001852.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Bjorn Kolbeck; Implementing Hyperscale and HPC Techniques and Technologies Necessary to Modern Media and Broadcast Workflows, SMPTE Meetings and Conferences ( October 2018); SMPTE, 2018, ISBN: 978-1-61482-960-7. Available at https://doi.org/10.5594/M001852

Snippet:

Bjorn Kolbeck; Implementing Hyperscale and HPC Techniques and Technologies Necessary to Modern Media and Broadcast Workflows, SMPTE Meetings and Conferences ( October 2018); SMPTE, 2018, ISBN: 978-1-61482-960-7. Available at https://doi.org/10.5594/M001852

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Bjorn Kolbeck; <cite>Implementing Hyperscale and HPC Techniques and Technologies Necessary to Modern Media and Broadcast Workflows</cite>, SMPTE Meetings and Conferences ( October 2018); SMPTE, 2018, ISBN: 978-1-61482-960-7. Available at <a href="https://doi.org/10.5594/M001852" target="_blank" rel="noopener">https://doi.org/10.5594/M001852</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Bjorn Kolbeck; <cite id="bib-10-5594-m001852">Implementing Hyperscale and HPC Techniques and Technologies Necessary to Modern Media and Broadcast Workflows</cite>, SMPTE Meetings and Conferences ( October 2018); SMPTE, 2018, ISBN: 978-1-61482-960-7
<span class="doi">10.5594/M001852</span>
</li>