AI-Led data center networking

The rapid growth of large language models (LLMs) and AI-driven workloads is revolutionizing data center (DC) networking, demanding unparalleled scalability, low latency and energy efficiency. Training LLMs involves massive datasets and computational challenges, requiring parallelization across thousands of GPUs. This necessitates advanced backend networking, leveraging protocols like RoCEv2 and InfiniBand for direct GPU-to-GPU communication, minimizing CPU bottlenecks. Additionally, new scale-up communications are going to be a large part of the AI DC network, and new protocols like Ultra Ethernet might become the key standard alternatives to InfiniBand.