Rambus Advances AI 2.0 with GDDR7 Memory Controller IP

April 23, 2024 -- As the latest addition to the Rambus portfolio of industry-leading interface and security digital IP for AI 2.0, the GDDR7 memory controller will provide the breakthrough memory throughput required by servers and clients in the next wave of AI inference.

Memory Solutions for AI 2.0

AI 2.0 represents the revolutionary world of generative AI. AI 2.0 leverages the enormous growth in Large Language Models (LLMs) and their kin to create new multimodal content. Multimodality means text, images, speech, music, video can be combined as inputs to create outputs in all these media and more. Examples include creating a 3D model from an image or a video from a text prompt.

LLMs have scaled to over a trillion parameters with data sets in the billions of samples. Training LLMs requires enormous computational power supported by the latest high-performance memory solutions.

Supercharging AI Inference with GDDR7

The output of the AI 2.0 training process is an inference model that can be employed to create new multimodal content from a user’s prompts. Since accuracy and fidelity increase with model size, there is an ongoing push to larger and larger inference models. And as AI inference becomes increasingly pervasive and moves out from the data center to the edge and endpoints, it drives the need for more powerful processing engines with tailored high-performance memory solutions across the entire computing landscape.

GPUs have been the inference engines of choice, and in the case of edge and endpoint applications, such as servers and desktops, these have been GPUs using GDDR6 memory. GDDR6, however, has reached the practical limit of standard NRZ signaling at 24 Gigabits per second (Gbps) data rates. To meet the bandwidth needs of future GPUs, a new generation of GDDR using a new signaling scheme is required. Enter GDDR7 memory which using PAM3 signaling can boost data rates to 40 Gbps and higher.

Rambus Silicon IP for AI 2.0

As the preferred silicon IP supplier for AI 2.0, Rambus offers industry-leading HBM, PCIe and CXL Controller IP and now the industry’s first GDDR7 Memory Controller IP. The Rambus GDDR7 Controller provides a full-featured, bandwidth-efficient solution for GDDR7 memory implementations. It supports 40 Gbps operation providing 160 Gigabytes per second (GB/s) throughput for a GDDR7 memory device, a 67% increase over the industry’s highest throughput GDDR6 Controller (also from Rambus). The Rambus GDDR7 Controller enables a new generation of GDDR memory deployments for cutting-edge AI accelerators, graphics and high-performance computing (HPC) applications.

“Delivering greater memory performance is mission critical as AI 2.0 workloads push bandwidth requirements higher than ever before,” said Neeraj Paliwal, general manager of Silicon IP, at Rambus. “With our breakthrough GDDR7 Controller IP solution, designers can quickly take advantage of this latest generation of GDDR memory at industry-leading throughput.”

“GDDR7 memory offers significant performance gains over GDDR6,” said Soo-Kyoum Kim, vice president, memory semiconductors at IDC. “The Rambus GDDR7 Controller IP solution will be a vital tool for anyone that wants to take advantage of the improved speed and latency features offered by GDDR7.”

Rambus GDDR7 Controller key features:

• Supports all GDDR7 link features including PAM3 and NRZ signaling
• Supports broad range of GDDR7 device sizes and speeds
• Optimized for high efficiency and low latency across a wide variety of traffic scenarios
• Flexible AXI interface support
• Low-power support (self-refresh, hibernate self-refresh, dynamic frequency scaling, etc.)
• Reliability, Availability and Serviceability (RAS) features – such as end-to-end data path parity, parity protection for stored registers, etc.
• Comprehensive memory test support
• Integration support for third-party PHYs available
• Validated utilizing the latest GDDR7 VIP and memory vendor memory models

The Rambus GDDR7 Memory Controller IP is available now. Learn more about the Rambus GDDR7 Controller here or download our white paper.