From Traces to Test: How the AI Industry Is Finally Speaking the Same Workload Language
Keysight is a co-author on MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces, presented at the Industry Track at MLSys 2026, one of the premier conferences showcasing the latest developments at the intersection of AI/ML and systems.
AI Fabric Validation Has a Fragmentation Problem
If you are responsible for validating an AI fabric before the cluster goes live, you have likely faced this question: what workload do you run?
Conventional tests use synthetic traffic with randomized source-destination pairs. ECMP hashing looks great, flows spread evenly, and the device passes with flying colors. Then real training runs start and the problems the synthetic suite never caught begin to surface.
AI training collectives like AllReduce, AlltoAll, and AllGather create highly structured, correlated flow patterns tied to model architecture and parallelization strategy. These do not appear in randomized traffic generators. Hyperscalers guard their real workload traces. Simulator vendors build around proprietary formats. The result is siloed optimizations, surprises at deployment, and a co-design cycle that moves slower than AI model innovation demands. This is the problem MLCommons Chakra was built to solve.
An Open Standard for AI Workload Representation
Chakra is an open ecosystem for AI workload representation, benchmarking, and co-design under MLCommons. At its core is a standardized execution trace (ET): a directed acyclic graph capturing compute, memory, and collective communication operations with their dependencies, without exposing proprietary model weights or data. Think of it as a portable behavioral fingerprint of a distributed AI workload that every ecosystem participant can read and act on.
The project originated at Meta and Georgia Tech in 2023, was adopted by MLCommons, and has grown to a 40-plus member working group. The MLSys 2026 paper, co-authored by NVIDIA, AMD, Meta, HPE, Scala Computing, Georgia Tech, Harvard, and Keysight, is the definitive community reference covering the schema, trace collection pipeline, downstream use cases, and real-world case studies.
Four Ways to Use a Chakra Execution Trace
Trace analysis examines real workload behavior at a fidelity synthetic benchmark cannot match, breaking down compute versus communication versus idle time across production LLM runs including GPT-3, Llama 3, Mixtral, and DeepSeek-MoE.
Trace replay re-executes operations on real hardware via PyTorch native backends without re-running the full training stack. One paper case study shows how replay identified parallelizable AllGather and ReduceScatter operations, yielding a 2x collective performance improvement.
Trace Simulation drives Chakra traces through full-system simulators to perform what-if analysis for diverse collective algorithms and/or next-generation network fabric architectures. Chakra natively runs on ASTRA-sim, an open-source distributed AI system simulator from Georgia Tech, that Keysight also actively contributes to.
Hardware emulation drives physical devices under test with Chakra-grounded traffic at full line rate. This is the most demanding downstream use case and where Keysight contributes to the paper.
How Keysight Turns a Workload Fingerprint Into a Hardware Test
Keysight AI Data Center Builder (KAI DCB) natively ingests Chakra execution traces and uses them to generate high-fidelity RDMA network traffic, which it then sends directly to the physical device under test — such as a switch fabric or NIC. Unlike software simulation, KAI DCB produces actual traffic that mirrors the AI collective communication patterns captured in the trace, putting real silicon through the same pressure it would face on the floor of a live AI cluster.
In the MLSys 2026 paper, we used KAI DCB to drive a four-switch 12.8T Clos fabric at 400 Gbps with an MoE workload stressing AllReduce and AlltoAll simultaneously. In isolation both collective types performed near line rate. When interleaved as in production, high-rate AllReduce flows triggered DCQCN congestion control, throttling the smaller AlltoAll flows and creating stragglers with measurable job completion time degradation. No synthetic suite would have found this. A Chakra-grounded hardware emulation did.
This is the shift-left imperative for AI infrastructure: surface emergent, system-level behavior at component validation time, before the cluster is built, not after.
What Comes Next and Why It Matters
Keysight is also working with the MLCommons Chakra community on InfraGraph, an emerging open representation of AI datacenter infrastructure. Paired with Chakra ETs, it would enable workload-aware topology design exploration and close the loop on the full SW/HW co-design cycle.
AI clusters are scaling rapidly, and architectures are growing more complex. Chakra gives the industry the shared workload language it has been missing, rigorous enough for production-grade emulation and open enough to be adopted without competitive friction. Keysight's participation reflects a straightforward conviction: the most credible validation tools are built on the same workload representation the co-design community uses, not a synthetic approximation of it.
MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces was presented at MLSys 2026 in Bellevue, WA. Paper is here https://arxiv.org/abs/2605.11333. Open-source code and traces: github.com/mlcommons/chakra. Learn more about KAI DCB at keysight.com.