PTP Driven Frame Sync for Multi-GPU Workflows

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Conference Paper
Content Type: Original Research
Volume: 2022, No. 10, pp. 1–16
Abstract: GPU-based video processing applications that composite rendered graphics with live video require synchronization of the GPU processing to the video essence stream timebase to prevent visual artifacts caused by temporal misalignment. One such application is in-camera visual effects (ICVFX) in virtual production, the technique of capturing GPU-rendered visual effects composited with live action in-camera within an LED volume. As the media industry continues to move from an SDI to a ST 2110 infrastructure, virtual production environments for ICVFX utilize ST 2059-2 PTP locked Network Interface Controllers (NICs) or Data Processing Units (DPUs) for accurate packet pacing of the media essence streams according to the ST 2110-21 specifications. — GPU processing of video frames is typically performed at the display refresh frequency with updates aligned to frame boundaries. This paper describes a solution for synchronizing GPU display updates with a ST 2059-2 PTP locked NIC, where the PTP Hardware Clock (PHC) is disciplined by the PTP stack running on the NIC/DPU. The NIC generates a Transistor-Transistor-Logic (TTL) signal at the required frequency which is then used to align the GPU's display engine frame boundaries with those of the PTP synchronized ST 2110-20 video essence streams to within nanosecond accuracy. Application Programming Interfaces (APIs) allow GPU-based video processing applications to align the generation and processing of frames across multiple GPUs, as well as perfectly synchronizing updates across all displays. — The ability to leverage the ST 2059-2 PTP profile used to synchronize the ST 2110 essence streams to additionally synchronize displays attached to GPUs, significantly simplifies the overall infrastructure deployment and management relative to traditional means of synchronizing displays using a Sync Pulse Generator (SPG) over coax infrastructure.
Publication Date: 2022-10-24
DOI: 10.5594/M001972
ISBN: Print: 978-1-61482-963-8
Link: https://doi.org/10.5594/M001972
Author(s): Thomas KernenNVIDIA
Thomas TrueNVIDIA
Ian WilliamsNVIDIA
Keyword(s): PTP, GPU, Data Processing Unit (DPU), NIC, ST 2110, Time & Sync, Timing, Virtual Production
Copyright: © 2022 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

Variable Frame Rate Technology — Change Is Good! [Active]
Tighter NIC/GPU Integration Yields Next Level Media Processing Performance [Active]
PTP hardware clock infrastructure for Linux https://www.kernel.org/doc/html/latest/driver-api/ptp.html EXTERNAL
“VESA Enhanced Extended Display Identification Data Standard”, Release A, Revision 2, September 25, 2006. EXTERNAL
Insider preview - Windows Time service in Windows Server 2019 https://docs.microsoft.com/en-us/windows-server/networking/windows-time-service/insider-preview EXTERNAL
E. Galan, “Use of Virtual Sets in the Broadcasting of Major Events”, Revista Latina de Comunicacion Social, January 2011, vol 66, pp 922-944. EXTERNAL
NVIDIA Bluefield Data Processing Unit https://www.nvidia.com/en-us/networking/products/data-processing-unit/ EXTERNAL
"IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems," in IEEE Std 1588-2008 (Revision of IEEE Std 1588-2002) , vol., no., pp.1-269, 24 July 2008, doi: 10.1109/IEEESTD.2008.4579760. EXTERNAL
M. Kavakli and C. Cremona, "The Virtual Production Studio Concept – An Emerging Game Changer in Filmmaking," 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2022, pp. 29-37, doi: 10.1109/VR51125.2022.00020. EXTERNAL
NVIDIA Quadro Sync II, https://www.nvidia.com/en-us/design-visualization/solutions/quadro-sync/ EXTERNAL
Linux testptp https://github.com/torvalds/linux/blob/master/tools/testing/selftests/ptp/testptp.c EXTERNAL
NVIDIA ConnectX-6Dx NIC https://www.nvidia.com/en-us/networking/ethernet/connectx-6-dx/ EXTERNAL
SMPTE OV 2110-0:2018 [Active]
SMPTE ST 2059-2:2020 [Active]
SMPTE ST 2110-10:2017 [Superseded]
SMPTE ST 2110-20:2017 [Superseded]
SMPTE ST 2110-21:2017 [Superseded]
SMPTE ST 2110-22:2022 [Active]

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001972.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Thomas Kernen, Thomas True, and Ian Williams; PTP Driven Frame Sync for Multi-GPU Workflows, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001972

Snippet:

Thomas Kernen, Thomas True, and Ian Williams; PTP Driven Frame Sync for Multi-GPU Workflows, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001972

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Thomas Kernen, Thomas True, and Ian Williams; <cite>PTP Driven Frame Sync for Multi-GPU Workflows</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at <a href="https://doi.org/10.5594/M001972" target="_blank" rel="noopener">https://doi.org/10.5594/M001972</a></span>

SMPTE's HTML Pub

Preview:

Snippet:

<li>
Thomas Kernen, Thomas True, and Ian Williams; <cite id="bib-10-5594-m001972">PTP Driven Frame Sync for Multi-GPU Workflows</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
<span class="doi">10.5594/M001972</span>
</li>