Loading...
Low-Overhead Intra-Host Container Communication With Hardware Offloading
Su, Qiang Niu, Zhixiong Shu, Ran Cheng, Peng Xiong, Yongqiang Han, Dongsu Xue, Chun Jason Xu, Hong
Su, Qiang
Niu, Zhixiong
Shu, Ran
Cheng, Peng
Xiong, Yongqiang
Han, Dongsu
Xue, Chun Jason
Xu, Hong
Supervisor
Department
Computer Science
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Containers are widely embraced for their deployment and performance benefits over virtual machines. Yet, for many data-intensive applications in containerized clouds, bulky data transfers may impose performance issues. In particular, communication across co-located containers on the same host incurs large overheads in memory copy and the kernel’s TCP stack. Existing solutions such as shared-memory networking and RDMA have their own limitations, including insufficient memory isolation and limited scalability. This paper presents PipeDevice, a new system for low overhead intra-host container communication. PipeDevicefollows a hardware-software co-design approach — it offloads data forwarding entirely onto hardware, which accesses application data in hugepages on the host, thereby eliminating CPU overhead from memory copy and TCP processing. PipeDevicepreserves memory isolation and scales well to connections, making it deployable in public clouds. Isolation is achieved by allocating dedicated memory to each connection from hugepages. To achieve high scalability, PipeDevicestores the connection states entirely in host DRAM and manages them in software. Evaluation with a prototype implementation on commodity FPGA shows that for delivering 80 Gbps across containers PipeDevicesaves 63.2% CPU compared to kernel TCP stack, and 40.5% over FreeFlow. PipeDeviceprovides salient benefits to applications. For example, we port baidu-allreduce to PipeDeviceand obtain ?2.2× gains in allreduce throughput.
Citation
Q. Su et al., "Low-Overhead Intra-Host Container Communication With Hardware Offloading," in IEEE Transactions on Networking, doi: 10.1109/TON.2024.3520210
Source
IEEE Transactions on Networking
Conference
Keywords
Containers, Kernel, Hardware, Throughput, Servers, Scalability, Cloud computing, Software, Field programmable gate arrays, Bridges
Subjects
Source
Publisher
IEEE