What is InfiniBand?
InfiniBand is a high-speed data transfer technology that was developed in the late 1990s as a solution to address the increasing demand for faster and more efficient communication between computer systems. It was specifically designed to provide a low-latency and high-bandwidth interconnect for server clusters and supercomputers.
At its core, InfiniBand is an input/output (I/O) architecture that enables data communication between devices, such as servers, storage systems, and network switches, over a high-speed serial link. It uses a switched fabric topology, where data is transmitted in packets across a network of InfiniBand switches.
One of the key features of InfiniBand is its ability to provide both remote direct memory access (RDMA) and message passing semantics. RDMA allows data to be transferred directly between the memory of two devices without the involvement of the operating system, reducing CPU overhead and improving overall system performance. Message passing, on the other hand, enables the exchange of data between processes running on different devices.
Basic Concepts of InfiniBand
1. InfiniBand Architecture
The InfiniBand architecture consists of three main components: Host Channel Adapters (HCAs), switches, and cables. HCAs are the interface cards installed in servers and storage systems, while switches connect these devices together to form a network. Cables are used to physically connect the HCAs and switches.
2. Ports and Lanes
In InfiniBand, each HCA and switch has one or more ports. Each port is further divided into multiple lanes, with each lane capable of transmitting data in both directions simultaneously. The number of lanes determines the overall bandwidth of the port.
3. Virtual Lanes
To ensure efficient data transmission, InfiniBand introduces the concept of virtual lanes. Virtual lanes allow the division of the physical lanes into different priority levels, ensuring that traffic of higher priority is given precedence over lower priority traffic. This enables the smooth and reliable communication of data across the InfiniBand network.
4. Subnet Manager
The subnet manager is a key component in an InfiniBand network. It is responsible for managing and maintaining the overall network structure, including assigning addresses to devices, managing routing tables, and ensuring optimal traffic flow.
5. InfiniBand Performance
InfiniBand delivers impressive performance capabilities, with data transfer rates ranging from 10 to 400 gigabits per second (Gbps) per port. The low latency and high bandwidth of InfiniBand make it ideal for demanding applications that require fast and efficient communication between nodes.
Overall, InfiniBand is a powerful high-speed data transfer technology that provides the foundation for high-performance computing and storage systems. By leveraging its low latency and high bandwidth capabilities, InfiniBand enables the efficient exchange of data across a network, improving the overall performance and productivity of modern computing environments.