3
As AI shifts from centralised training to distributed inference, network architecture is being redrawn around proximity, latency and resilience. Metro fiber, edge connectivity, and optical infrastructure will become as critical to AI as compute itself.
The economics of artificial intelligence (AI) are definitively moving away from AI training towards AI production or inference and the real-time use of AI models. While training often represents huge upfront capital expense, inference is where this investment starts to deliver financial returns. AI service providers have a vested interest in moving from training to inference as quickly as possible.
As AI models evolve from development to deployment, the architecture is also fundamentally shifting. We are entering the era of distributed AI inference, a transition that is not just changing where AI processing happens, but also redrawing the network map.
The managed AI inference market reached $23.1 billion in 2025, surpassing the $16.3 billion training market, and is projected to reach $106.8 billion by 2030. Inference is also expected to account for roughly two-thirds of AI compute in 2026, up from one-third in 2023.
The decentralisation of AI infrastructure – diverging fiber strategies
This macro-economic shift from AI training to inference heralds a divergence in physical layer design. Real-time AI applications demand proximity to the user. We are seeing inference infrastructure increasingly distributed across metro environments, enterprise locations, smart campuses, autonomous factories, and healthcare systems.
It is clear that a single network architecture for both types of AI workloads will not suffice.
AI training – the density imperative
AI training requires compute nodes to minimize the time it takes to synchronize parameters across thousands accelerators or graphics processor units (GPUs). This demands full, non-blocking east-west network fabrics in a spine-leaf architecture. At the fiber level, the focus is wholly on extreme density in a concentrated data center campus.
Training data centers are built in large greenfield sites in remote areas with easy access to cheap, abundant power. Fiber is heavily focused internally (inter-rack and inter-building campus interconnects) in the back-end network, rather than needing external carrier diversity. For the AI training process:
– Facilities are set up at points of concentrated power used to support the back-end network.
– High bandwidth fiber connections are needed to optimize synchronization between Accelerators in the back-end network.
– Compute is less latency sensitive.
To support this, DC architects are turning to advanced connectivity technologies like Very Small Form Factor (VSFF) connectors and ultra-high density, high-fiber-count intermittently bonded ribbon (IBR) cables, to maximize pathway utilization.
AI inference – the distribution imperative
Inference networks, by contrast, bypass the expensive east-west backend fabric almost entirely. Since inference instances operate more independently (or in much smaller pods), the infrastructure focus shifts to robust north-south connectivity in order to get the response back to the user as quickly as possible. The infrastructure strategy relies heavily on diverse metro fiber rings, cloud on-ramps, and carrier hotel interconnection.
Inference processes individual user queries rather than large synchronized runs. Fiber infrastructure is optimized for high-capacity external connectivity rather than internal east-west mesh type network fabrics. For the AI inference process:
– Processing needs to be closer to users.
– Low latency is needed to manage latency dependent applications.
– Must integrate with cloud infrastructure.
– Located in or near major metro areas to minimize network latency.
The passive fiber infrastructure inside and between data centers differs significantly based on latency requirements, cluster architecture, and bandwidth density. This edge-centric model ensures that AI can interact with the physical world in real-time, whether it is a transport system managing traffic flows or a robotic assembly line executing split-second adjustments. But this geographic distribution comes with unprecedented infrastructure demands.
For training, latency is a synchronization bottleneck inside the data center, solved by dense high-performance fiber and fewer network hops. For inference, latency is a user-experience bottleneck outside the data center.
Inference flips the script on connectivity
With the growth of inference, AI operators can no longer build static networks. As AI applications scale, network design must prioritise greater metro and edge capacity, robust route diversity, and infrastructure that can expand without repeated redesign or service disruption.
Constructing these resilient, high-capacity metro rings requires advanced cabling solutions. Deploying intermittently bonded ribbon cables, for example, allows for rapid, ultra-high-fibre-count installations within congested metro conduits, ensuring that operators can build the necessary route diversity to guarantee uptime.
Furthermore, as the latency tolerances for real-time edge AI shrink to microseconds, network architects must begin looking toward next-generation optical connectivity. Integrating multi-core fibre (MCF) or even deploying hollow-core fibre (HCF) on critical, highly sensitive routes offers a pathway to significantly reduce signal delay and future-proof the network.
Connectivity is the new compute
Historically, compute and storage were the primary constraints on enterprise technology. In the distributed AI era, connectivity will become just as critical as the compute itself.
It does not matter how powerful an AI accelerator sits at the edge if the network cannot deliver the data fast enough or if a single fibre cut takes the system offline. Network readiness; built on dense, scalable, and ultra-reliable optical infrastructure – will increasingly determine the performance, resilience, and scalability of real-time AI applications.
As we look toward 2030, the network map will look vastly different. The operators and enterprises that recognise the shifting demands of AI inference today and build their passive and active network infrastructure to match will be the ones leading the next generation of digital transformation
