Definition: A type of stacked DRAM designed for extremely fast data transfer rates by minimizing distance between memory and GPU.
Explanation
High Bandwidth Memory (HBM) is a form of stacked DRAM placed very close to the GPU die, often just millimeters away. It features wide buses and short electrical traces to minimize data travel distance, enabling terabytes per second transfer speeds. This design significantly reduces latency and energy consumption compared to traditional memory architectures, making it ideal for workloads that require rapid and repeated access to large datasets, such as AI training.
Example
In AI training, where billions of model weights are updated repeatedly, HBM provides the necessary low latency and high bandwidth to handle massive data throughput efficiently, enabling faster model convergence and reduced energy costs.
Who This Is For
HBM is primarily for hardware engineers, AI researchers, and system architects working on high-performance computing, especially in fields requiring intensive data processing like machine learning training and advanced graphics rendering.
Related Terms
DRAM, GPU, AI training, memory hierarchy, latency, bandwidth
Also Known As
HBM