Flowlet Load Balancing for AI Workloads
Modern AI training workloads produce highly bursty and phase‑synchronized traffic because of collective communication primitives such as All‑Reduce. These traffic characteristics frequently undermine conventional ECMP‑based load balancing, causing persistent path collisions and inefficient utilization of network resources. In contrast, packet‑level load balancing can alleviate imbalance but introduces packet reordering, which requires special handling at the NIC and introduces the risk of retransmissions.
Flowlet-based load balancing offers promising middle ground. By leveraging natural gaps between communication bursts, flowlets enable dynamic path re-selection without inducing packet reordering.
In this post, we intentionally focus on a baseline evaluation of flowlet‑based load balancing under a symmetric Clos fabric with equal‑cost paths. Our goal is not to provide a comprehensive evaluation across all deployment scenarios, but rather to establish a clean and interpretable baseline that isolates the interaction between bursty AI traffic and flowlet behavior.
The figure illustrates how flowlet based load balancing redistributes traffic across equal cost paths in response to burst boundaries, compared to static ECMP hashing.
The topology represents a Clos / ECMP fabric with multiple equal-cost paths between the sender (connected to the left leaf switch) and receivers (connected to the right one). The colored blocks labeled 1 and 2 denote different flows or communication phases generated by a synchronized AI workload.
Before flowlet reselection, ECMP hashes entire flows onto fixed paths. When multiple bursty flows arrive in a phase‑synchronized manner, they are likely mapped onto the same path, leading to persistent collisions, queue buildup, and underutilization of other available paths.
The arrow indicates a natural inter‑burst gap that exceeds the flowlet timeout. At this boundary, the next burst is treated as a new flowlet, allowing the switch to reselect a forwarding path.
After reselection, subsequent bursts are redistributed across different equal‑cost paths. Because reassignment occurs only at burst boundaries, packet reordering is avoided, while path utilization becomes more balanced, and congestion pressure is reduced.
A symmetric Clos fabric with equal cost paths is used to isolate flowlet behavior under bursty AI traffic, avoiding confounding effects from path asymmetry or incast.
We deliberately restrict the evaluation to this baseline configuration to avoid confounding factors such as incast contention or heterogeneous path conditions. This controlled setup allows us to focus exclusively on how burst boundaries translate into flowlets and how these flowlets are mapped onto equal‑cost paths.
The workload emulates a representative AI communication phase by explicitly controlling burst size and inter‑burst gap, capturing the synchronized and bursty nature commonly observed in collective operations. As a reference point, a non‑bursty continuous transmission pattern is also evaluated to distinguish effects caused by burst‑induced flowlet formation from those inherent to the network topology.
All experiments operate at full link rate, ensuring that path utilization, queue build up, and scheduling effects manifest clearly even under idealized and balanced conditions.
Heating figure reports the per path traffic distribution across equal cost paths under different load balancing schemes and traffic patterns in the baseline symmetric topology. An ideal outcome corresponds to an even split of 12.5% traffic per path.
Under bursty traffic conditions, ECMP exhibits pronounced load imbalance. For both 256 µs batch experiments with synchronized bursts, ECMP consistently maps multiple bursts onto the same path, resulting in some paths carrying 25% of the total traffic while others remain idle. This behavior reflects persistent hash collisions rather than transient fluctuations, highlighting ECMP’s sensitivity to phase‑synchronized burst arrivals.
In contrast, flowlet‑based load balancing achieves near‑ideal path utilization under the same bursty conditions. When burst boundaries introduce natural inter‑burst gaps, flowlets allow forwarding paths to be reselected across bursts. As a result, traffic is evenly distributed across all equal‑cost paths, with per‑path utilization closely clustered around the ideal 12.5%, regardless of whether burst staggering is applied.
The behavior differs under non‑bursty (LINE) traffic patterns. In the absence of clear burst boundaries, flowlet formation becomes less stable, leading to uneven traffic distributions in some cases. This observation underscores that the effectiveness of flowlet‑based load balancing is tightly coupled to the temporal structure of the workload.
Overall, these results should be interpreted as baseline behavior in a symmetric setting. They demonstrate that flowlet‑based load balancing naturally aligns with the bursty and phase‑synchronized structure of AI collective communication, where well‑defined burst gaps enable safe and effective path re‑selection without packet reordering.
At the same time, the simplified setting considered here serves as a reference point rather than a definitive performance bound. Understanding baseline behavior is a necessary first step toward reasoning more complex and realistic deployments.
While this post focuses on a baseline symmetric scenario by design, several important directions remain for future exploration. These include evaluating flowlet‑based load balancing under asymmetric path conditions, incast‑heavy communication patterns, and different congestion control mechanisms such as PFC and DCQCN. Extending the analysis to larger‑scale clusters and higher link speeds is also essential for assessing robustness and practicality in production of AI fabrics.