SRv6 for Deterministic Path Placement in AI Backends

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 6 October 2025.¶

Copyright Notice

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

Hyperscale AI training clusters rely on massive GPU-to-GPU data exchanges, where synchronization delays caused due to congestion delays and packet loss directly impact model convergence time and operational costs.¶

These workloads generate large, predictable flows that require ultra-low latency, high bandwidth, and precise congestion control to maintain efficiency. Traditional networking approaches, such as ECMP-based per-flow load balancing, suffer from poor entropy due to the limited number of RoCEv2 flows, leading to fabric hotspots, congestion, and slow reconvergence after failures.¶

SRv6 uSID (NEXT-CSID) provides the ability to steer in the fabric, allowing the NIC (i.e., SmartNIC, DPU) to perform deterministic path placement of ROCEv2 traffic through the fabric. This ensures predictable performance, fine-grained traffic control, and real-time adaptation to congestion in a stateless manner. ¶

Future revisions of this draft will cover additional use-cases (multi-path transport, stateless interaction between AI/LLM leasing a cluster infra and the operator managing the cluster, etc).¶

The document draft-filsfils-srv6-dc-frontend-wan explains how SRv6 uSID (NEXT-CSID) is applied to a converged DC Frontend and WAN fabric.¶

2. Terminology

SRv6

Segment Routing over IPv6 [RFC8986].¶

uSID

Micro-segment. Formally defined as NEXT-CSID in [I-D.ietf-spring-srv6-srh-compression].¶

The term uSID (micro SID) predates the formal naming and has been widely adopted across the industry - including operators with large-scale deployments, vendors, open-source implementations, and used consistently in multi-vendor interoperability reports.¶

To maintain alignment with the formal specification while also acknowledging the widespread and practical use of the term, this document uses uSID and NEXT-CSID interchangeably.¶

ECMP

Equal-Cost Multi-Path¶

uN

The uN is a short notation for the End behavior with NEXT-CSID, PSP, and USD flavors as defined in [I-D.ietf-spring-srv6-srh-compression].¶

uA

The uA local behavior is a short notation for the End.X behavior with NEXT-CSID, PSP, and USD flavors [I-D.ietf-spring-srv6-srh-compression].¶

ROCEv2

RDMA over Converged Ethernet version 2 [IBTA-ROCEv2].¶

NIC

Network Interface Card, a hardware component that connects a computer to a network.¶

SmartNIC

A Network Interface Card with embedded processing capabilities, designed to offload network and storage tasks from the host CPU.¶

DPU

Data Processing Unit, a specialized processor designed to offload and accelerate data-centric tasks, often used in network and storage functions.¶

GPU

Graphics Processing Unit, a processor designed for rendering graphics and performing parallel computation tasks, commonly used for AI and machine learning workloads.¶

3. AI Traffic Characteristics and Challenges

AI workloads exhibit highly structured traffic patterns:¶

• Predictable Elephant Flows: Collectives' communications require multiple GPUs to exchange data in a structured manner that is known in advance. Flows between GPUs are large, long-lived, high throughput and predictable.¶
• Synchronized Bursts: Model synchronization causes periodic, coordinated traffic spikes.¶
• Low ECMP Entropy: Data exchange between GPUs relies on a small number of flows (ROCEv2 Queue Pairs), leading to poor performance of traditional load-balancing solutions. A 5-tuple based ECMP load-balancing results in non-homogenous utilization across the fabric, leading to congestion.¶
• Resilience: Network failures prolong training time and increase significantly operational costs. Therefore, it is imperative for the network to provide high resiliency, and fast reaction to congestion.¶

4. SRv6 for Deterministic Path Placement

SRv6 enables the NIC to directly control the AI workload traffic journey through the fabric by encoding an ordered list of segments in the packet header.¶

• AI Scheduler: Upon AI job orchestration, the collectives' communications are defined (i.e., the GPU Topology). The AI scheduler determines the optimal fabric routed paths based on all the running jobs in the fabric, and the GPU topology for each one of them.¶

  • The encoding of a path as an SRv6 Network Program does not require any per-path communication between the AI Scheduler and the fabric.¶
  • At fabric bring up, the controller managing the fabric communicates the overall topology together with SRv6 uSID (NEXT-CSID) explicit instructions for each link (uA). These instructions are statically configured are thus independent of any routing protocol dynamic state. The AI Scheduler build any path through the fabric without any further control-plane interaction with the routers.¶

• NIC: The NIC, before sending the ROCEv2 traffic, encapsulates with an outer IPv6 header and encodes in the packet header the sequence of instructions to enforce the precomputed path through the fabric.¶

  • Note that an outer IPv6 header allows to encode 6 uSIDs in the Destination Address. This implies that even upon presence of a super-spine in a 3-tier Clos fabric, the entire path can be encoded without the need of any additional Segment Routing extension Header (SRH).¶
• Highly Scalable Stateless Fabric: The routers in the fabric enforce the path by following the sequence of SRv6 instructions in the packet header. There is no per-flow state in the network (unlike MPLS RSVP-TE which would require the instantiation of states in the fabric on a per GPU-to-GPU deterministic path basis).¶
• Congestion Feedback Loop: The NICs react in real time to congestion notifications (ECN, inband latency measurement, Packet Trimming, inband packet loss). These mechanisms are preserved and leveraged by the solution to optimize traffic steering and prevent congestion hotspots. At any time (without any fabric signaling or dependency), within a few nanoseconds, the NIC can change the deterministic path through the fabric by simply changing the outer IPv6 Destination Address. The change is only at the source NIC. There is no change required at any of the intermediate devices in the fabric.¶

5. Illustration

The following figure depicts a typical 2-tier Clos topology.¶

          Spine4                      Spine5
            |                           |
   +--------+----+--------------+-------|-----+
   |             |              |       |     |
   |   +---------|---+----------|---+---+-----|----+
   |   |         |   |          |   |         |    |
+--+------+   +--+------+    +--+------+   +--+------+
|  Leaf1  |   |  Leaf2  |    |  Leaf3  |   |  Leaf4  |
+----+----+\ /+----+----+    +----+----+\ /+----+----+
     |      X      |              |      X      |
     |     / \     |              |     / \     |
     |    /   \    |              |    /   \    |
+----+----+   +----+----+    +----+----+   +----+----+
|  DPU1   |   |  DPU2   |    |  DPU3   |   |  DPU4   |
|    |    |   |    |    |    |    |    |   |    |    |
|  GPU1   |   |  GPU2   |    |  GPU3   |   |  GPU4   |
+---------+   +---------+    +---------+   +---------+

The topology consists of two Spine devices. Each of the Spines is connected to four Leaf devices.¶

There are 4 NICs, which are connected through the host interface (e.g., PCIe) to a GPU. In this example each NIC is dual-homed to two Leaf devices.¶

6. Benefits

• Deterministic Path Placement: SRv6 allows the NIC to control the path of each flow through the fabric.¶
• Minimum-MTU: A plain outer IPv6 encapsulation allows to encode 6 uSIDs in the outer DA. This implies that without the need of additional extension headers, only with 40Bytes of IPv6 encapsulation, we can encode up to 6 intermediate waypoints allowing to enforce a path in a 3-tier Clos network. This is sufficient to control a path hop-by-hop (link by link) through a leaf, spine, super-spine, spine, leaf.¶
• Congestion Feedback Loop: Instant rerouting at the source based on ECN, in-band measured One-Way and Two-Way latency, Packet Trimming feedback and in-band packet loss, without any dependency of routing protocols. There is neither any control-plane signaling involved between the GPU and the fabric, nor between the AI orchestrator and the fabric devices.¶
• Standardization: Open, vendor-agnostic implementation¶
• Ease of operation: as opposed to black-box proprietary solution which packs opaque layer-2 optimization, the SRv6 solution is minimalistic, IP based, fully standardized and a rich ecosystem (vendor, merchant and open source). The deterministic and open nature of the solution simplifies troubleshooting.¶

7. Hyperscale

AI workloads are deployed across thousands of GPUs in multi-tier Clos networks, requiring a networking architecture that scales efficiently. SRv6 uSID (NEXT-CSID) ensures deterministic path placement while maintaining scalability through the following mechanisms:¶

• Stateless Fabric: Unlike RSVP-TE or MPLS-TE, which require per-flow state on network devices, SRv6 enforces paths by including all the instructions in the packet header. This eliminates state explosion as the number of GPUs increases.¶
• uSID Encapsulation: The SRv6 uSID (NEXT-CSID) encoding allows paths to be efficiently encoded even in multi-tier topologies, reducing encapsulation overhead while supporting large deployments. If more than 6 instructions are required, a simple IPv6 Segment Routing Extension Header can be used to encode additional instructions.¶
• Cross-Datacenter Extension: The same SRv6-based mechanism can extend beyond a single cluster to multi-datacenter AI fabrics (i.e., inter-DC AI training), where deterministic path placement ensures efficient inter-cluster data transfers.¶
• Overlay Tenant Separation: SRv6 can provide per-tenant network segmentation, ensuring AI workloads from different tenants or jobs are isolated while sharing the same physical infrastructure. By adding into the network program VPN Service SIDs; traffic steering and resource allocation can be enforced at the network level without requiring additional overlay encapsulations.¶

8. Security Considerations

The deployment model described in this document is secured leveraging the mechanisms defined in [RFC8986].¶

9. Acknowledgements

The authors would like to recognize the work of Lihua Yuan, Guohan Lu, Rita Hui, and Riff Jiang at Microsoft.¶

Rita Hui presented this use-case at MPLS & SRv6 World Congress in March 2025. A recording is available here: https://www.segment-routing.net/conferences/Paris25-Microsoft-Rita-Hui/¶

The authors would like to acknowledge the work of the developers who have enabled this use-case in the open-source [SONiC] implementation. In particular: Carmine Scarpitta, Abhishek Dosi, Changrong Wu, Kumaresh Perumal, Eddie Ruan, and Yuqing Zhao.¶

Authors' Addresses