Home » 3D-Parallelism

Habana Developer Blog

/ 3D-Parallelism
One of the main challenges in training Large Language Models (LLMs) is that they are often too large to fit on a single node or even if they fit, the training may be too slow. To address this issue, their training can be parallelized across multiple Gaudi accelerators (HPUs).
Sign up for the latest Habana developer news, events, training, and updates.