OCI RDMA Superclusters in GenAI Applications
Introduction:
As I began exploring Oracle's GenAI offerings, I encountered recently popularized RDMA Superclusters in Oracle’s Cloud Infrastructure (OCI). Given my background in computer science sparked my interest in this technology. I would say that the computer architecture supporting RDMA Superclusters have certainly transformed the landscape of AI infrastructure. This post aims to shed some light on RDMA Superclusters by summarizing their key features, and for my Oracle database compatriots, comparing them with the well-known Oracle Database Real Application Clusters (RAC) architecture. I'll also delve into the impressive performance capabilities of OCI’s RDMA Superclusters, highlighting their potential for high-speed, low-latency communication in various applications. What I have found is that OCI’s RDMA Supercluster architecture implementation is quite impressive, and, I would say, OCI in general provides a robust ecosystem for Generative AI workloads.
**Sidebar personal history note: Back in the mid 1980’s I ran Oracle v4 on a DEC Rainbow 8086 PC with 8-megahertz clock speed and 640K of memory (upgraded from the standard 128K for a couple thousand dollars) with a whopping 20 meg hard drive (upgraded from the standard 5 meg). It still blows my mind that computers now a days are in the Ghz clock speeds. BTW: we thought we were really doing some high performance computing (on a PC) with an 8087 math co-processor that one could use, for example, with a special Fortran compiler.
What are RDMA Superclusters?
RDMA stands for Remote Direct Memory Access. RDMA Superclusters are high-performance computing clusters that use the RDMA technology for communication between nodes. RDMA allows direct memory access from one computer to another without involving the operating system of either computer. This architecture, combined with RDMA technology, significantly reduces latency and improves throughput compared to traditional networking technologies. Oracle’s key to scaling to tens of thousands of GPUs is wrapped up in the nonblocking network fabric.
In an RDMA Supercluster, each node typically has its own memory and processors, and communication between nodes is done directly through RDMA-enabled network interfaces. This setup is commonly used in applications requiring high-speed data transfer and low-latency communication, such as training AI models containing trillions of parameters, scientific computing, data analytics, and large-scale simulations.
RDMA Superclusters can be challenging to set up and configure due to the specialized networking hardware and software required. However, OCI simplifies this process for the customer by handling the heavy lifting for you. With OCI's streamlined setup, superclusters become more accessible, offering significant performance benefits for applications that can leverage RDMA's low-latency, high-throughput capabilities, such as model training and fine-tuning.
Is the RDMA architecture similar to Oracle database RAC architecture?
From a database architecture perspective, many DBA’s have experience with and knowledge of the RAC architecture. Those of us unfamiliar with RDMA supercluster architecture may find it interesting to compare and contrast this with a familiar multi-node architecture: Oracle Database Real Application Clusters (RAC). The short story is, they are optimized for different use cases, with Oracle RAC being more focused on traditional database workloads and RDMA Super Clusters being more suited for high-performance computing applications.
Architecture Overview:
Oracle RAC: Oracle RAC is a clustered database architecture that allows multiple database instances to access a single database, sharing access to data files by synchronizing over the private cluster interconnect network. It provides high availability and scalability by distributing the workload across multiple nodes.
RDMA Super Clusters: RDMA Super Clusters are high-performance computing clusters that use RDMA technology for communication between nodes. Each node typically has its own memory and processors, and communication between nodes is done directly through RDMA-enabled network interfaces.
Communication Model:
Oracle RAC: In Oracle RAC, instances communicate with each other using a cluster interconnect, which is a private network dedicated to inter-instance communication. The interconnect can be based on Ethernet or specialized high-speed interconnects.
RDMA Super Clusters: RDMA Super Clusters use RDMA technology for communication between nodes. RDMA allows for direct memory access from one node to another without involving the operating system, which can reduce latency and improve throughput.
Scalability:
Oracle RAC: Oracle RAC provides scalability by allowing additional instances to be added to the cluster as needed, distributing the workload across the new instances.
RDMA Super Clusters: RDMA Super Clusters can also provide scalability by adding more nodes to the cluster. Each additional node can contribute to the overall computing power and storage capacity of the cluster.
Performance:
Oracle RAC: Oracle RAC can provide good performance for applications that require high availability and scalability. However, the performance can be impacted by the overhead of managing a clustered environment.
RDMA Super Clusters: RDMA Super Clusters are designed for high-performance computing applications that require low-latency communication between nodes. The use of RDMA technology can significantly improve performance for these types of applications.
Use Cases:
Oracle RAC: Oracle RAC is well-suited for applications that require high availability and scalability, such as online transaction processing (OLTP) systems.
RDMA Super Clusters: RDMA Super Clusters are more suitable for high-performance computing applications, such as scientific computing, data analytics, and large-scale simulations, that require low-latency communication between nodes.
RAC vs RDMA Architecture Summary:
In summary, Oracle RAC and RDMA Super Clusters are both clustered architectures designed to improve scalability and availability. However, they are optimized for different use cases, with Oracle RAC being more focused on traditional database workloads and RDMA Super Clusters being more suited for high-performance computing applications.
RDMA Performance
OCI has RDMA Super Clusters with as low as 2 microsecond network latency (i.e. the time it takes for a data packet to travel from one point to another in a network; typically measured in milliseconds (ms) or microseconds (µs).)
Why is 2 microsecond latency important?
Normal network latency can vary widely depending on factors such as the distance between nodes, the quality of the network infrastructure, and the load on the network. In a local area network (LAN), latency is often in the range of microseconds to milliseconds. In wide area networks (WANs), latency can be higher, ranging from milliseconds to tens or hundreds of milliseconds.
Low network latency is important for applications that require real-time or near-real-time communication, such as online gaming, financial trading, and high-performance computing. Low latency allows for faster data transfer and more responsive applications.
OCI’s RDMA Super Cluster network latency is impressive!
Oracle Cloud Infrastructure's claim of achieving as low as 2 microseconds of network latency for their RDMA Super Clusters is impressive. This ultra-low latency can significantly benefit applications that require high-speed, low-latency communication between nodes.