Congestion Control in AI Networks
Most network engineers treat congestion as a simple "too much traffic" problem. In AI data centers, that oversimplification can cost you millions of dollars in wasted GPU compute.
Here's what is actually happening.
AI traffic behaves nothing like web traffic. Collective operations like All-Reduce cause every GPU in a training job to transmit simultaneously — the instant backpropagation completes. There's no natural smoothing. Just a synchronized burst hitting the network like a hammer.
This creates two fundamentally different congestion types, each demanding a different solution:
1. ECMP Hash Collision — Multiple flows land on the same uplink while others sit idle, dragging effective bandwidth down to ~60%. Fix: per-packet or flowlet-based hashing to spread flows evenly across all available paths.
2. Many-to-One Incast — In All-to-All operations (the dominant communication pattern in MoE models), every GPU sends to every other GPU at once. No matter how evenly you distribute the paths, all traffic must eventually converge on the destination's downlink port. Dynamic routing cannot help here — you have to slow the senders down.
These two problems are not interchangeable. Conflating them is a design error.
PFC and the Victim Flow Problem
Start with why PFC exists: RoCE is a lossless protocol. Any dropped packet triggers a retransmission, and every retransmission is effective bandwidth stolen directly from your training throughput. PFC (Priority Flow Control) is the last line of defense — when a receiver buffer is close to exhaustion, the switch sends a PAUSE frame upstream, hop-by-hop, halting all traffic on the same priority queue and forcing every sender to stop. Zero packet loss, preserved.
But PFC is a blunt instrument. It has no idea which flow caused the congestion — it can only slam the brakes on the entire priority queue. That blindness has a hidden cost.
When Sources 1, 2, and 3 flood Destination 1 and exhaust the Leaf port buffer, PFC fires. The backpressure propagates upstream and pauses all traffic on that priority — including Source 4, which is sending completely unrelated traffic to Destination 2. Source 4 did nothing wrong.
This is Head-of-Line Blocking: zero packet loss for the congesting flows, at the expense of throttling the innocent ones. In a cluster where hundreds of GPUs share the same fabric, this cascading effect spreads like dominoes — making overall training throughput unpredictable.
DCQCN
The answer to HOL Blocking is to rate-limit the flows causing congestion before PFC ever needs to fire — not wait until the buffer is nearly full and then hit everything at once. That's exactly what DCQCN (Data Center Quantized Congestion Notification) does, built on an ECN/CNP feedback loop. What has changed is where it executes and how fast:
- The switch marks packets with ECN CE (Congestion Experienced) once queue depth crosses a configured threshold
- The receiver generates a CNP (Congestion Notification Packet), delivered at strict priority
- The sender's SuperNIC rate-limits only the specific Queue Pair causing congestion — all other flows are untouched
The critical innovation: the entire process runs in SuperNIC ASIC hardware, not OS kernel software. Reaction time compresses from milliseconds to microseconds, handling millions of congestion events per second, completely transparent to the GPU.
When ECN and congestion control respond fast enough, PFC becomes the safety net you never need to pull.
Buffer: The Overlooked Third Pillar
Traditional switches statically carve buffer into fixed slices per port and traffic class (TC). Too little and you drop packets; too much and you waste capacity. AI's bursty traffic pattern makes this design even worse.
Dynamic shared buffer flips the logic: all ports and TCs draw from a single shared pool, borrowing capacity on demand when bursts arrive instead of immediately dropping or triggering PFC. Per-flow and per-tenant limits prevent elephant flows from monopolizing the pool and starving everything else.
Buffer also plays a critical role in the congestion control loop itself. The ECN/CNP feedback cycle has inherent latency — from the switch marking CE, to the receiver generating a CNP, to the sender actually slowing down. Packets keep arriving throughout that window and need somewhere to land. Shared buffer is that cushion, buying the control loop enough time to act before drops occur.
Buffer sizing is also physically constrained by distance. When a PFC PAUSE frame is in transit, the upstream sender keeps transmitting — unaware it should stop. Those in-flight packets must be absorbed somewhere. This reserved headroom is called PFC Headroom:
PFC Headroom = Link Rate × RTT
Light travels through fiber at ~200,000 km/s (⅔ of c), giving ~5 μs of one-way delay per kilometer.
Example: 400GbE link, 2 km between switches → RTT = 2 × 2 km × 5 μs = 20 μs
Headroom = 400 Gbps × 20 μs = 1 MB
Headroom scales linearly with distance — which is why large-scale AI fabrics demand far more buffer capacity than typical campus networks.
Three thresholds, three lines of defense: crossing the ECN threshold triggers DCQCN rate reduction; crossing the PFC threshold triggers hop-by-hop pause; crossing the buffer limit means drops. The entire architecture is designed to keep traffic on the safe side of the first line.
Per-packet or flowlet load balancing handles path imbalance. Congestion control handles incast. Shared buffer absorbs bursts and buys time for the control loop. Each is necessary. None can substitute for the others.
If you're designing or evaluating AI network infrastructure, understanding which problem each mechanism actually solves matters far more than chasing headline port speeds.
Want to know where congestion is hitting your cluster hardest? Start by looking at your PFC trigger rate.