Device Memory Sharing Study
Metadata
- Publisher
- SMPTE — Pasadena, CA
- Doc Type
- Conference Paper
- Content Type
- Original Research
- Volume
- 00, pp. 1–35
- Abstract
- As broadcast systems continue their transition from dedicated hardware appliances to standard commercial off-the-shelf (COTS) computing platforms, the efficiency of data movement becomes a critical factor in overall system performance. In particular, the growing reliance on Graphic Processing Unit (GPU)-accelerated processing for real-time media encoding, transcoding, AI, and image manipulation demands optimized data exchange both within compute nodes and across distributed systems. This shift introduces new challenges in memory sharing and communication, especially when dealing with heterogeneous memory hierarchies and high-throughput, low-latency requirements. Using internode [1] Remote Direct Memory Access (RDMA) transfer as a baseline: This study presents a comparative evaluation of memory sharing mechanisms across three distinct GPU transfer paths: inter-node GPU-GPU, intra-node host-GPU and intra-node GPU-GPU memory exchange. The performance of each path is assessed under different software configurations: native memory operations without the aid of any communication framework - commonly referred to as native, and higher-level abstractions, like [2] Unified Communication X (UCX) and [3] Libfabric. These configurations represent a range of abstraction levels and transport optimizations commonly used in High-Performance Computing (HPC) and media processing workloads. All experiments are performed on a uniform hardware platform featuring GPUs with programmable interfaces such as [4] Compute Unified Device Architecture (CUDA) enabling developers to offload and execute code directly on NVIDIA GPU. The system is equipped with network interfaces that support RDMA allowing for low-latency, high-throughput data transfers. Key performance metrics, including [5] PCle (Peripheral Component Interconnect Express) bandwidth, latency and Central Processing Unit (CPU) utilization, are measured to evaluate the impact of each communication method on data transfer efficiency. These results are intended to support the development of high-performance applications that require efficient and cost-effective memory sharing between computational devices, particularly in domains such as media processing, machine learning, and scientific computing where data locality and transport efficiency are essential. In addition, this study highlights how higher-level communication frameworks provide flexible abstraction.
- Publication Date
- 2025-10-13
- ISBN
- Electronic:
978-1-61482-966-9 - Author(s)
- Jonas OhlandMichael LefebvreThomas J. TrueGareth Sylvester-BradleySithideth Viengkhou
- Keyword(s)
- GPU, Libfabric, UCX, COTS, Media Exchange Layer (MXL), RDMA, CUDA
- Copyright
- © 2025 SMPTE
Bibliographic Reference(s)
- R. Recio, B. Metzler, P. Culley, J. Hilland and D. Garcia, “RFC 5040: A Remote Direct Memory Access Protocol Specification,” [Online]. Available: https://www.rfc-editor.org/info/rfc5040 EXTERNAL
- UCF Consortium, “Unified Communication X,” [Online]. Available: https://openucx.org/ EXTERNAL
- OpenFabrics Allianc, “Libfabric: Open Fabric Interface,” May 2023. [Online]. Available: https://github.com/ofiwg/libfabric EXTERNAL
- NVidia Corporation, “CUDA Toolkit Documentation 13.0,” [Online]. Available: https://docs.nvidia.com/cuda/ EXTERNAL
- Synopsys Inc, “What is PCI Express (PCle)?,” 2025. [Online]. Available: https://pcisig.com/ EXTERNAL
- DMF-MXL, “MXL: Media eXchange Layer,” July 2025. [Online]. Available: https://github.com/dmf-mxl/mxl EXTERNAL
- NVidia Corporation, “GPUDirect RDMA,” [Online]. Available: https://developer.nvidia.com/gpudirect EXTERNAL
- Wikipedia, “Infiniband,” [Online]. Available: https://en.wikipedia.org/wiki/lnfiniBand EXTERNAL
- IBM, “RDMA Over Converged Ethernet (RoCE) version 2,” July 2025. [Online]. Available: https://www.ibm.com/docs/en/aix/7.3.0?topic=access-rdma-over-converged-ethernet-roce-version-2 EXTERNAL
- Cisco Systems, ““Ethernet Technologies”. Internetworking Technology Handbook,” December 2018. [Online]. Available: https://web.archive.org/web/20181228005303/http://docwiki.cisco.com/wiki/Ethernet_Technologies EXTERNAL
- N. Manchanda and K. Anand, “Non-Uniform Memory Access (NUMA,” 27 01 2014. [Online]. Available: http://www.cs.nyu.edu/-lerner/spring10/projects/NUMA.pdf EXTERNAL
- https://en.wikipedia.org/wiki/Transmission_Control_Protocol, “Wikipedia,” 2025. [Online]. Available: https://en.wikipedia.org/wiki/Transmission_Control_Protocol EXTERNAL
- Wikipedia, “Wikipedia,” 2025. [Online]. Available: https://en.wikipedia.org/wiki/User_Datagram_Protocol EXTERNAL
- OpenFabrics Alliance, “OpenFabrics Alliance, Training Materials: Programming with the Verbs API,” [Online]. Available: https://www.openfabrics.org/training EXTERNAL
- loc.gov, “V210,” [Online]. Available: https://www.loc.gov/preservation/digital/formats/fdd/fdd000353.shtml EXTERNAL
- Network Lessons, “Network Lessons. “Introduction to Precision Time Protocol (PTP)”,” [Online]. Available: https://networklessons.com/ip-services/introduction-to-precision-time-protocol-ptp EXTERNAL
- Government of Canada, “Network Time Protocol (NTP),” [Online]. Available:: https://networklessons.com/ip-services/introduction-to-precision-time-protocol-ptp EXTERNAL
- M. Kerrisk, “Linux manual page,” 17 05 2025. [Online]. Available: https://man7.org/linux/man-pages/man2/perf_event_open.2.html EXTERNAL
- AMD 5.1, “AMD.com,” [Online]. Available: https://www.amd.com/en/developer/uprof.html#:~:text=AMD%20uProf%20(%E2%80%9CMICRO%2Dprof,Instinct%E2%84%A2%20MI%20Series%20accelerators EXTERNAL
- Proxmox, “Proxmox Virtual Environment,” 2025. [Online]. Available: https://www.proxmox.com/en/ EXTERNAL
- Linux RDMA open source project, “RDMA Core Userspace Libraries and Daemons,” [Online]. Available: https://github.com/linux-rdma/rdma-core/blob/master/Documentation/libibverbs.md EXTERNAL
- Linux RDMA, “rdma-core,” [Online]. Available: https://github.com/linux-rdma/rdma-core/blob/master/Documentation/librdmacm.md EXTERNAL
- DMF-MXL, “MXL Fabric API,” [Online]. Available: https://github.com/dmf-mxl/mxl/pull/78 EXTERNAL
- Mellanox, “mlnx-toolsV24.10.1,” 2025. [Online]. Available: https://github.com/Mellanox/mlnx-tools/blob/mlnx_ofed/python/mlnx_qos EXTERNAL
- ofiwg, [Online]. Available: https://ofiwg.github.io/libfabric/main/man/fi_shm.7.html]. EXTERNAL
- NVIDIA, [Online]. Available: https://developer.download.nvidia.cn/compute/DevZone/docs/html/C/doc/html/group_CUDART_MEMORY_g36b9fe28f547f28d23742e8c7cd18141.html EXTERNAL
- NVIDIA, [Online]. Available: https://developer.download.nvidia.com/compute/DevZone/docs/html/C/doc/html/group_CUDART_MEMORY_g48efa06b81cc031b2aa6fdc2e9930741.html EXTERNAL
- ofiwg, [Online]. Available: https://github.com/ofiwg/libfabric/issues/11407 EXTERNAL
- Intel, “Intel Omni-Path,” [Online]. Available: https://www.intel.com/content/dam/www/public/us/en/documents/presentation/omni-path-fabric-software-architecture-overview.pdf EXTERNAL
- AWS, “AWS EFA,” [Online]. Available: [https://aws.amazon.com/hpc/efa/]. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/978-1-61482-966-9-21.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; Device Memory Sharing Study, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9
Snippet:
Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; Device Memory Sharing Study, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9
HTML (ISO 690 compliant)
Preview:
Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; Device Memory Sharing Study, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9
Snippet:
<span class="citation">Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; <cite>Device Memory Sharing Study</cite>, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9</span>
SMPTE's HTML Pub
Preview:
Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; Device Memory Sharing Study, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9
Snippet:
<li> Jonas Ohland, Michael Lefebvre, Thomas J. True, Gareth Sylvester-Bradley, and Sithideth Viengkhou; <cite id="bib-978-1-61482-966-9-21">Device Memory Sharing Study</cite>, MTS 2025, Article 21 (pp. 1 to 35); SMPTE, 2025, ISBN: 978-1-61482-966-9 </li>