AI Infrastructure Scaling Domains
In my earlier post on scaling AI infrastructure, I explained how AI systems expand across the stack, from chip to board, server, rack, cluster, data center, and edge. The next question is: as these systems grow, what new capabilities emerge, and how does performance multiply?
Unlocking this scale means solving shifting challenges: feeding unprecedented compute density in silicon, turning networks into synchronized compute paths for thousands of GPUs, and conquering latency to deliver real-time AI at the edge.
AI Infrastructure Scaling Domains
The diagram below provides a simple way to understand how scaling opportunities evolve across the AI stack. As infrastructure expands, the source of performance moves from compute and memory inside the package to interconnects, fabrics, optics, and distributed systems. These AI Scaling Domains help identify where the next gains can be unlocked, how the engineering problem changes with scale, and why validation must adapt as AI systems grow.
Viewed through this lens, AI infrastructure can be organized into five scaling domains:
- Scale In: Scaling within the silicon and package domain, including pre-silicon design, chiplets, memory, and die-to-die/package interconnects.
- Scale Up: Scaling at rack level through node- and rack-level expansion.
- Scale Out: Scaling at cluster level through rack-to-rack data center fabric.
- Scale Across: Scaling at regional level through multi-site, Data Center Interconnect (DCI), and cloud interconnect fabric.
- Scale Above: Scaling AI to the distributed edge and beyond terrestrial limits via space-based data centers, and global satellite links.
Scale In: Design AI-ready silicon and chiplets
This domain focuses on silicon and package optimization, including pre-silicon architecture, chiplet-based designs, high-bandwidth memory integration, and advanced die-to-die and package-level interconnects.
Key priorities include achieving first-silicon success, maintaining robust signal integrity, power integrity, and timing at higher data rates, improving photonic integrated circuit performance for high-speed transceivers, and integrating UCIe, HBM, and chiplets with proven multi-PHY interoperability.
Scale Up: Maximize rack-level bandwidth with next-gen interconnects
This domain focuses on node- and rack-level expansion through tightly coupled accelerator fabrics, such as large GPU or TPU domains linked by low-latency, high-bandwidth switched interconnects.
The priorities are to accelerate validation of 800G, 1.6T, and 3.2T modules and links, move lane speeds to 224 Gbps and 448 Gbps while improving cost per bit and power per bit, ensure conformance with IEEE 802.3, OIF-CEI, UALink, PCIe, and CXL, and demonstrate multi-vendor interoperability.
Scale Out: Prove AI fabric performance at data-center scale
This domain centers on cluster-level scaling across high-performance rack-to-rack fabrics designed for AI east-west traffic.
The goal is to maximize GPU utilization, deliver predictable job completion time under real workloads, maintain stable and lossless fabrics through disciplined congestion control, qualify Short Reach (SR) and Long Reach (LR) optics for reliable links, and streamline component and architecture choices.
Scale Across: Extend AI bandwidth across regions
This domain focuses on regional and multi-site connectivity through Data Center Interconnect (DCI), metro and long-haul optical fabrics, and connectivity to resources in the cloud.
The priorities are validating high-capacity coherent modules and related devices, including 800ZR and 1600ZR, assuring transport resilience under load and faults, enabling cost-efficient regional interconnects and metro DCI, and optimizing optical line system components such as muxes, filters, amplifiers, and new optical fiber types.
Scale Above: Assure planetary-scale AI over satellite, wireless & edge
This domain covers AI infrastructure beyond terrestrial data centers, spanning space-based compute such as orbital data centers and satellite constellations, AI-RAN (Artificial Intelligence Radio Access Network), and distributed edge AI. Together, these layers form a connected architecture designed to extend capacity, reduce latency, and enable seamless global connectivity.
The priorities are enabling low-latency edge inference over Non-Terrestrial Networks (NTN), 5G, and 6G, scaling network capacity for AI workloads, validating new 6G air-interface features, predicting link performance with accurate models, and supporting future architectures that extend compute into space.
Why This Matters
As AI scales from silicon to the global edge, performance bottlenecks do not just shift; new opportunities emerge at each stage. Every domain brings distinct engineering and validation challenges, along with new ways to expand bandwidth, improve utilization, reduce latency, extend reach, and unlock new system capabilities. For teams building next-generation AI infrastructure, success depends on the ability to understand and validate each of the five domains with confidence. This is where Keysight helps, connecting insight from silicon to system to network so customers can reduce risk, unlock performance, and scale AI with greater confidence.
Learn more about Scaling AI Data Centers: click here to read the eBook and click here to download the Poster to see how validation across the stack helps teams design, build, and scale AI infrastructure with confidence.