The GB300 NVL72 integrates 72 GPUs and 36 CPUs in a single liquid-cooled rack — every GPU needs 800 Gbps of scale-out bandwidth to reach its full potential

1. The GB300 NVL72 Revolution: What Changes for AI Networks

The NVIDIA GB300 NVL72 represents a generational leap in AI compute density. A single rack integrates 72 Blackwell Ultra GPUs and 36 Grace CPUs connected via 5th-generation NVLink, delivering 130 TB/s of non-blocking in-rack bandwidth and up to 1,080 PFLOPS of sparse FP4 tensor compute. Compared to the Hopper-based HGX H100, the GB300 NVL72 delivers 70x more AI FLOPS, 1.5x more HBM3e memory (288 GB per GPU, 40 TB per rack), and 2x the networking bandwidth at 14.4 TB/s.

What makes the GB300 era fundamentally different for network architects is the ConnectX-8 SuperNIC, which provides 800 Gbps of scale-out connectivity per GPU. A full NVL72 rack therefore requires up to 72 x 800G = 57.6 Tbps of aggregate scale-out bandwidth — an order of magnitude beyond what traditional data center networks were designed to handle. For organizations building AI infrastructure, understanding these requirements is covered in depth in our AI infrastructure data center cabling requirements guide.

The practical implication: NVIDIA reports that GB300 NVL72 delivers 10x user responsiveness (tokens per second per user) and 5x throughput per megawatt compared to Hopper. This is achieved not just through silicon improvements but through the co-design of compute, interconnect, and networking — a design philosophy explored in our analysis of AI frontend vs backend network design differences.

2. 200G Lossless Ethernet: The Scale-Out Backbone

While NVLink handles in-rack GPU-to-GPU communication at 130 TB/s, the scale-out network — connecting multiple NVL72 racks, storage arrays, and management infrastructure — relies on Ethernet or InfiniBand operating at 200G per lane. The 200G-per-lane SerDes is the foundational building block: an 800G port is constructed from four 200G lanes (4x200G), and a 1.6T port from eight. This per-lane speed is what enables the 800 Gbps per-GPU connectivity that ConnectX-8 SuperNICs deliver.

Spectrum-X Ethernet vs Traditional Ethernet

NVIDIA's Spectrum-X Ethernet platform is purpose-built for AI workloads, offering adaptive routing, per-flow congestion management, and enhanced RoCEv2 support that standard Ethernet switches lack. Unlike traditional data center Ethernet — which was designed for best-effort, many-to-many traffic — Spectrum-X implements lossless Ethernet semantics that guarantee zero packet drop for RDMA traffic, making it functionally equivalent to InfiniBand for AI training while maintaining Ethernet's cost and interoperability advantages.

According to industry data, RoCEv2-based Ethernet solutions saw 31% shipment growth in 2025, compared to 19% for InfiniBand — signaling a structural shift toward Ethernet-based AI fabrics. The global optical interconnect market, valued at under $20 billion in 2025, is projected to reach $92 billion by 2028 with a 65% compound annual growth rate, driven primarily by data center AI networking demand. For organizations evaluating their connectivity supply chain, our fiber optic cable supplier selection guide provides a structured evaluation framework.

Rail-optimized topology assigns each GPU to a dedicated leaf switch, eliminating congestion hotspots that plague traditional spine-leaf designs

Why 200G Lossless, Not Just 200G

Standard Ethernet is a best-effort protocol — it drops packets under congestion and relies on TCP retransmission for reliability. For web traffic, this is acceptable. For AI training, it is catastrophic. A single dropped packet during an all-reduce gradient synchronization operation can stall the entire collective for milliseconds, wasting thousands of GPU-seconds of compute. Lossless Ethernet eliminates this by guaranteeing that priority traffic classes never experience packet loss, using a combination of flow control and congestion notification mechanisms described in Section 4.

3. Rail-Optimized Topology: Designing for GPU-to-Storage

Traditional data center networks use a spine-leaf topology where all leaf switches connect to all spine switches, and traffic between any two endpoints may traverse multiple hops through shared switches. This works well for enterprise workloads with many small flows, but it creates two critical problems for AI training: congestion hotspots when multiple large flows contend for the same spine port, and unpredictable latency that destabilizes collective communication patterns.

The Rail-Optimized Alternative

Rail-optimized topology — also called rail-attached or rail-aligned design — assigns each GPU position in a rack to a dedicated leaf switch (a "rail"). In an NVL72 with 8 GPUs per compute tray, there are 8 rails, each with its own dedicated leaf switch and spine switch. Cross-rack traffic for tensor-parallel or expert-parallel operations follows a single, deterministic path through a matched spine, ensuring that:

• Hop count is minimized — most GPU-to-GPU traffic crosses at most one spine hop
• Head-of-line blocking is eliminated — no shared queues between unrelated traffic classes
• Congestion is predictable — each rail's bandwidth is dedicated, not shared
• Failure domains are isolated — a single rail failure affects only its assigned GPUs

In practice, a GB300 cluster with 8 NVL72 racks (576 GPUs) might deploy 8 rail switches, each handling 72 x 800G links per rack — a total of 576 x 800G connections per rail. This demands high-radix switches (64+ ports of 800G) and high-density optical panels. Our patch panel cable management guide for data centers covers the physical infrastructure strategies needed to manage these fiber counts.

4. Lossless Mechanisms: PFC, ECN, and RoCEv2 Deep Dive

Three complementary mechanisms transform standard Ethernet into a lossless fabric suitable for AI workloads. Understanding their interaction is essential for any network architect designing GB300-scale infrastructure.

Priority Flow Control (PFC — IEEE 802.1Qbb)

PFC extends standard Ethernet pause frames to operate per priority class rather than on all traffic. Ethernet defines 8 priority classes (0-7); in a typical RoCEv2 deployment, class 3 is designated as the "lossless" class for RDMA traffic. When a switch's receive buffer for class 3 approaches capacity, it sends a PFC pause frame to the upstream port, which temporarily stops transmitting class 3 traffic while allowing other classes to flow normally. This prevents buffer overflow and the resulting packet drops — but only if configured correctly.

Common PFC misconfigurations include enabling PFC on too many priority classes (which defeats the purpose of per-class control), setting pause thresholds too high (causing drops before pause takes effect), or failing to configure headroom buffers (which can lead to pause frame oscillation and throughput collapse).

Explicit Congestion Notification (ECN — RFC 3168)

While PFC reacts to buffer pressure reactively, ECN provides proactive congestion signaling. Switches mark the ECN field (IP header DSCP bits) on packets passing through queues that exceed a configured threshold. When the receiving endpoint detects ECN-marked packets, it signals the sender to reduce its transmission rate via RoCEv2 Congestion Notification (CN) packets. This creates a closed-loop feedback system that:

• Throttles senders before buffers fill and PFC pauses are triggered
• Reduces the frequency and duration of PFC pauses
• Provides fairness among flows sharing the same priority class
• Enables adaptive rate control for long-lived RDMA connections

RoCEv2: RDMA over Converged Ethernet

RoCEv2 (RDMA over Converged Ethernet version 2) enables zero-copy, kernel-bypass remote memory access over standard Ethernet. A GPU can directly read from or write to another GPU's HBM3e memory across the network without involving either system's CPU or operating system — the same semantic that NVLink provides within a rack. This is critical for:

• Gradient synchronization in data-parallel training (all-reduce operations)
• Parameter server communication in model-parallel training
• NVMe-oF storage access for checkpoint I/O and dataset loading
• GPU-to-GPU peer access for pipeline-parallel training across racks

PFC, ECN, and RoCEv2 form a three-layer lossless stack — PFC prevents drops, ECN prevents pauses, and RoCEv2 enables zero-copy GPU-to-GPU memory access

5. Scale-Up vs Scale-Out: The NVLink-Ethernet Boundary

One of the most critical architectural decisions in GB300 deployment is understanding where NVLink (scale-up) ends and Ethernet (scale-out) begins. This boundary determines which communication patterns stay within a rack at 130 TB/s and which traverse the network at 800 Gbps per GPU.

Scale-Up: NVLink 5th Generation

Within the NVL72 rack, 5th-generation NVLink provides 1.8 TB/s of bandwidth per GPU, creating a 130 TB/s non-blocking fabric where any GPU can reach any other GPU in a single hop. NVLink operates with memory semantics — GPUs can directly load and store to peer HBM3e memory via the fabric, with reductions executed inside the switch hardware during transit. This eliminates software-level data movement overhead and achieves near-zero latency for in-rack collective operations.

For training workloads like DeepSeek-V3 671B, NVIDIA reported achieving 1,648 TFLOPs per GPU using Megatron Core on GB300 NVL72 with 256 GPUs — a world record for MoE pre-training. This performance is only achievable because tensor-parallel and expert-parallel communication stays within the NVLink domain, while data-parallel and pipeline-parallel traffic crosses to the Ethernet domain.

Scale-Out: Spectrum-X Ethernet or Quantum-X800 InfiniBand

When workloads exceed a single NVL72 rack (72 GPUs), scale-out networking takes over. Each GPU connects to the scale-out fabric via its ConnectX-8 SuperNIC at 800 Gbps. The fabric choices are:

For most enterprises, Spectrum-X Ethernet is the pragmatic choice — it provides lossless semantics comparable to InfiniBand while leveraging the broader Ethernet ecosystem for interoperability, cost efficiency, and talent availability. For organizations that need deeper background on networking protocol choices, our analysis of Fibre Channel networking market trends provides context on how storage protocols are converging.

Scale-Across: Data Center Interconnect

For multi-site AI factories, scale-across networking uses 400ZR/ZR+ coherent optics over dark fiber or DWDM systems. Cloud service providers' direct purchases of WDM systems surged 60% in 2025, driving 400ZR/ZR+ pluggable module shipments up more than 50% quarter-over-quarter. This layer connects geographically distributed GB300 clusters for disaster recovery, workload migration, and distributed training across metro and long-haul distances.

6. AI Storage Network Optimization: NVMe-oF and Checkpointing

AI training workloads are not just GPU-compute-bound — they are increasingly storage-I/O-bound. As model sizes grow from billions to trillions of parameters, checkpointing, dataset loading, and weight distribution place enormous demands on the storage network. A GB300 NVL72 rack with 40 TB of GPU memory needs to checkpoint and restore that data in minutes, not hours.

NVMe over Fabrics (NVMe-oF) over RoCEv2

The most efficient storage protocol for AI workloads is NVMe-oF over RoCEv2, which extends NVMe's direct memory access semantics across the lossless Ethernet fabric. This allows GPUs to read training data and write checkpoints directly from NVMe SSD arrays without CPU intervention, achieving storage latencies as low as 10-20 microseconds over short-reach links.

The storage tiering hierarchy for a GB300 deployment typically follows:

7. Optical Infrastructure: Transceivers and Cabling for GB300

The physical layer of a GB300-ready network is where architecture meets reality. Every 800G scale-out port requires either an optical transceiver with fiber cabling, a direct-attach copper cable (DAC) for short distances, or an active optical cable (AOC) for moderate reach. The choices made here directly impact cost, power consumption, and reliability.

Transceiver Selection at 200G/400G/800G

For guidance on selecting the right transceiver form factors and lifespans, our article on optical transceiver lifespan and replacement covers failure modes and procurement strategies. For within-rack DAC cabling, our DAC cable types and use cases guide explains the trade-offs between passive copper, active copper, and AOC solutions.

Fiber Cabling: MTP/MPO for High-Density Scale-Out

A single GB300 NVL72 rack with 72 GPUs at 800 Gbps each requires 72 x 800G optical connections. At 8 fibers per 800G port (using 100G/lane), that is 576 fiber strands minimum — a density that demands MTP/MPO or MMC high-density connectors rather than individual LC duplex patch cables. A single 1U MPO panel can house 24-48 MPO connectors, each carrying 12 or 16 fibers, for up to 768 fibers in 1 rack unit.

For the fiber types used in these links, OM4 multimode is the standard for 100m-reach 800G SR8 links, while OS2 singlemode is used for 500m-reach 800G DR8 and 2km-reach 800G FR4 links. Our comprehensive fiber optic cable types guide (OS2, OM3, OM4, OFNR, OFNP) and our overview of different fiber optic cable types provide detailed selection criteria for each application.

A single GB300 rack may require 576+ fiber strands — MTP/MPO high-density panels are not optional, they are survival gear

Market Context for Optical Procurement

The optical components market is experiencing unprecedented demand pressure. According to Citi research, global optical interconnect shipments will grow from 110 million units in 2025 to 300 million units by 2028, with data center scenarios accounting for 89% of that volume. CPO (co-packaged optics) is expected to begin commercial volume production in 2027, with 1.8 million units shipped that year, scaling to 5.6 million by 2028. Organizations planning GB300 deployments should secure optical component supply chains 12-18 months ahead of deployment, as advanced optical chip capacity is projected to lag demand by 5-15% through 2026.

8. Migration Checklist: Preparing Your Data Center for GB300

Transitioning to GB300-ready infrastructure is not a network upgrade — it is a facility redesign. The following checklist covers the eight domains that must be addressed before the first NVL72 rack arrives.

For organizations concerned about installation quality and common fiber optic deployment pitfalls, our guide on common fiber optic installation problems provides field-tested troubleshooting advice that applies directly to GB300-scale deployments.

Key Questions Answered

Q1: What is NVIDIA GB300 NVL72 and how does it impact data center network design?

The NVIDIA GB300 NVL72 is a liquid-cooled rack-scale system integrating 72 Blackwell Ultra GPUs and 36 Grace CPUs with 130 TB/s of NVLink bandwidth. Each GPU receives 800 Gbps of scale-out connectivity through ConnectX-8 SuperNICs, demanding 200G or higher lossless Ethernet for rack-to-rack and rack-to-storage traffic. This redefines data center network design by requiring dedicated rail switches, high-density optical cabling, and lossless RoCEv2 transport for AI training and inference workloads.

Q2: Why is 200G lossless Ethernet critical for AI storage networks?

200G lossless Ethernet provides the per-lane bandwidth needed to saturate each GPU's 800 Gbps scale-out port without packet loss. Lossless transport via PFC and ECN ensures that RDMA-based NVMe-oF storage traffic and gradient synchronization data arrive without retransmission delays, which would otherwise stall GPU compute pipelines and degrade training throughput.

Q3: How does rail-optimized topology differ from traditional spine-leaf?

Rail-optimized topology assigns each GPU in a rack to a dedicated leaf switch (rail), ensuring that cross-rack traffic for tensor-parallel or expert-parallel operations takes a single hop through a matched spine switch. Traditional spine-leaf topology routes traffic through shared switches, creating congestion hotspots and unpredictable latency. Rail-optimized designs reduce hop count, eliminate head-of-line blocking, and simplify congestion management for AI workloads.

Q4: What is the difference between scale-up and scale-out networking in GB300 deployments?

Scale-up networking uses 5th-generation NVLink within the GB300 NVL72 rack, providing 1.8 TB/s per GPU and 130 TB/s total non-blocking bandwidth for in-rack GPU-to-GPU communication. Scale-out networking extends connectivity between racks using ConnectX-8 SuperNICs at 800 Gbps per GPU over Spectrum-X Ethernet or Quantum-X800 InfiniBand. Scale-up handles tensor-parallel and expert-parallel traffic; scale-out handles data-parallel, pipeline-parallel, and storage traffic.

Q5: How do PFC and ECN enable lossless Ethernet for AI workloads?

Priority Flow Control (PFC, IEEE 802.1Qbb) pauses traffic on specific priority classes without blocking other classes, preventing packet drop during congestion bursts. Explicit Congestion Notification (ECN, RFC 3168) marks packets at switch queues approaching capacity, signaling endpoints to reduce transmission rates proactively. Together, PFC and ECN create a lossless fabric that supports RoCEv2, enabling zero-copy GPU-to-GPU and GPU-to-storage data transfers essential for AI training.

Q6: What optical modules and cabling does a GB300 cluster require?

A GB300 NVL72 cluster typically uses 800G QSFP-DD or OSFP transceivers for scale-out connections (4x200G or 8x100G lane configurations), 400G transceivers for storage network links, and DAC or AOC cables for short-reach within-rack connections under 3 meters. MTP/MPO or MMC high-density fiber panels are essential for managing the massive fiber counts — a single GB300 rack with 72 GPUs at 800 Gbps each requires up to 72 x 800G optical links, equivalent to 1,152 fiber strands at minimum.

Q7: Can existing 100G or 400G infrastructure support GB300 workloads?

Existing 100G infrastructure cannot natively support GB300's 800 Gbps per-GPU scale-out requirement. 400G infrastructure can partially support GB300 by bundling multiple 400G links per GPU, but this increases latency, complicates congestion management, and wastes switch port density. A phased migration approach — upgrading spine layers to 800G first while maintaining 400G leaf layers — can bridge the gap, but full GB300 performance requires 800G end-to-end.

Q8: What is the total cost of ownership for a GB300-ready network infrastructure?

A GB300-ready network infrastructure typically adds 30-40% to the compute rack cost, covering 800G switches, 200G/400G/800G transceivers, high-density MTP/MPO fiber panels, and AOC/DAC cables. Power and cooling infrastructure for a single GB300 NVL72 rack can exceed 120 kW, requiring liquid cooling. However, the 50x AI factory output improvement over Hopper-based platforms means the cost per inference token drops significantly, delivering favorable TCO for organizations running sustained AI workloads.

AMPCOM Technical Team

Industry experts with 17+ years in enterprise network infrastructure and structured cabling systems