| Internet-Draft | ONCO: Problem, Use Cases, Requirements | July 2026 |
| Tan, et al. | Expires 7 January 2027 | [Page] |
Distributed artificial intelligence (AI) computing is increasingly deployed across geographically dispersed AI data centers (AIDCs) to meet the scale and performance demands of modern AI workloads. In such environments, the efficiency of distributed training and inference depends critically on tight coordination between optical transport networks and compute orchestration systems. However, today's infrastructure operates with isolated control planes: optical networks lack awareness of dynamic compute requirements, while compute schedulers have no visibility into real-time network conditions such as latency, bandwidth, or congestion. This decoupling leads to suboptimal resource utilization, degraded job performance, and inefficient scaling.¶
This draft presents the problem statement, outlines two representative use cases: distributed AI training and distributed AI inference, and specifies the requirements for Optical Networks and AI Computing Orchestration (ONCO). The goal is to enable bidirectional awareness, joint resource abstraction, and synchronized control across the compute-optical boundary, thereby supporting intent-driven, end-to-end provisioning of AI services over wide-area optical infrastructures.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 7 January 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The rapid development of large-scale AI applications, particularly those involving distributed training and inference across geographically dispersed AIDCs, has exposed a critical gap in today's infrastructure: the lack of coordination between optical transport networks and compute orchestration systems. While optical networks provide the high-bandwidth, low-latency, and deterministic connectivity required for wide-area AI computing collaboration, their control planes remain largely agnostic to the dynamic, heterogeneous demands of AI workloads. Conversely, compute schedulers operate without visibility into the underlying network's real-time state, such as path latency, available bandwidth, or congestion levels.¶
This decoupling leads to suboptimal resource utilization, degraded job performance, and inefficient scaling of distributed AI jobs. For example, a training job may be scheduled across distant AIDCs with abundant GPU resources but poor optical connectivity, resulting in prolonged synchronization phases and significant compute efficiency loss. Similarly, latency-sensitive inference services may be routed through paths that meet compute criteria but violate network service-level objectives.¶
To address these challenges, ONCO enables bidirectional awareness, joint resource abstraction, and synchronized control across the compute-optical boundary. This draft describes sample usage scenarios that drive ONCO requirements and will help to identify candidate solution architectures and solutions.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This document uses the following terms:¶
+------------------------------------------+
| AI Compute Service Request |
| (e.g., Distributed Training Job) |
+--------------------+---------------------+
|
+----------v----------+
| |
v v
+------------------+ +---------------------+
| Compute | | Optical Network |
| Scheduler | | Controller |
+------------------+ +---------------------+
| |
| |
v v
+------------------+ +-----------------+
| AIDC-A |<--Low-BW-->| AIDC-B |
| GPU: High |(e.g., 100G)| GPU: High |
| Load: Low | | Load: Low |
+--------+---------+ +---------+-------+
| |
| |
| High-BW (e.g., 400G) |
+<------------------------------>+
| |
| |
| High-BW (e.g., 400G) |
+<------------------------------>+
| |
+--------v---------+ |
| AIDC-C |<--------------------+
| GPU: Low |
| Load: High |
+------------------+
¶
The primary challenge lies in the management and control isolation between the computing domain (e.g., AI training clusters, cloud pools) and the optical transport network. This separation creates a "chasm" that prevents holistic resource optimization.¶
The optical network control plane operates without awareness of the real-time characteristics and requirements of the compute jobs it carries. Crucially, the physical layer properties of the optical infrastructure directly determine the feasibility and efficiency of AI workloads.¶
The physical distance between geographically distributed AIDCs imposes a fundamental propagation delay (roughly 5 microseconds per kilometer). For large-scale distributed training, this inherent latency directly impacts the time of global synchronization operations (e.g., All-Reduce). Furthermore, traditional packet-switched networks may introduce variable jitter, queuing delays, and potential packet loss. These can lead to frequent retransmissions and high synchronization penalties, causing massive GPU idle cycles and severely degrading compute efficiency.¶
The optical underlay possesses unique physical-layer capabilities, such as wavelength or time-slot switching, which can provision hard-isolated and deterministic circuits. These circuits eliminate jitter and guarantee a stable, latency-bounded channel. This is a critical differentiator for synchronizing massive GPU clusters across long distances. Without this physical-layer awareness, the control plane cannot leverage these unique deterministic features to support high-performance AI workloads.¶
Conversely, the compute orchestration layer (e.g., Kubernetes schedulers, AI job managers) makes resource allocation and job placement decisions based primarily on local compute and storage metrics (e.g., GPU availability, memory). It lacks visibility into the underlying network's physical state, including the end-to-end propagation delay and whether the available optical paths can provide the required deterministic performance. This results in compute jobs being scheduled across locations with poor physical connectivity (e.g., long-distance links with excessive latency or shared channels lacking isolation). Such scheduling causes significant communication bottlenecks and degraded overall job performance (i.e., compute efficiency loss).¶
This bidirectional lack of awareness creates a fundamental mismatch. While the optical network possesses the physical-layer capabilities (deterministic latency, hard-isolation) to act as the ideal transport for wide-area AI workloads, these capabilities are not exposed to or understood by the compute domain. Consequently, neither domain can adapt to the needs of the other, severely limiting the potential of wide-area collaborative computing.¶
Compounding the control isolation is the absence of a unified framework for evaluating the joint efficiency of compute and network resources.¶
Today, network performance is evaluated using traditional metrics like bandwidth utilization, latency, and packet loss. Compute performance is assessed through metrics like FLOPS (Floating Point Operations Per Second), job completion time, and resource utilization (CPU/GPU/memory). These evaluation systems each operate independently.¶
There is no standardized method to quantify the combined cost and benefit of a joint compute-and-network resource allocation decision. For instance, as shown in Figure 1, it is difficult to answer whether allocating a more powerful but distant GPU (with higher network latency) is more efficient than a less powerful but local one for a specific AI job.¶
This lack of a common evaluation language prevents the development of truly optimal co-scheduling algorithms that balance compute power against network quality.¶
Existing frameworks, such as Computing-Aware Traffic Steering (CATS)[I-D.ietf-cats-framework] [I-D.ietf-cats-usecases-requirements], have initiated efforts to integrate computing metrics into network routing decisions. However, these approaches are primarily designed for traffic steering at the network layer and fall short of the orchestration requirements for AI workloads over optical infrastructures. Specifically, two fundamental limitations exist.¶
First, existing frameworks cannot provision optical links on demand. CATS focuses on selecting an optimal path among already existing and established network routes. It lacks the capability to interact with the optical control plane to create new physical or logical links. This limitation becomes critical in cross-regional AI scheduling scenarios.¶
For example, a distributed training job spanning Beijing and Shanghai (1000 km apart) requires a high-bandwidth, low-latency, and deterministic optical channel to synchronize model parameters efficiently. The physical propagation delay over 1000 kilometers is a hard constraint, but the more unpredictable issue lies in packet jitter and queuing delays introduced by traditional IP-based forwarding. CATS-based traffic steering over existing shared pipes cannot guarantee the required deterministic performance, leaving the training job vulnerable to synchronization stalls and severe GPU utilization degradation.¶
In contrast, the optical physical layer can establish hard-isolated channels through dedicated wavelength or time-slot circuits (e.g., OXC or fgOTN). These channels offer deterministic latency, zero jitter, and guaranteed bandwidth, which are essential for efficient all-reduce operations over long distances. The existing framework cannot trigger this on-demand optical provisioning, which represents a fundamental gap.¶
Second, existing frameworks do not support full lifecycle orchestration of computing resources. They are generally limited to steering service requests to service instances that are already available. They cannot perform explicit reservation, locking, and subsequent release of high-performance resources (like GPU clusters).¶
In the Beijing-Shanghai scenario, effective orchestration cannot merely "steer" traffic. It must first jointly reserve compute resources at both AIDCs and then trigger the optical layer to provision a end-to-end deterministic optical containers to deliver the training job. Without this joint capability, both networks and computing remain in isolated silos, which prevents the consistent, high-performance execution of distributed AI workloads.¶
The growing scale and distribution of artificial intelligence workloads have created new demands on wide-area optical infrastructure, particularly in scenarios that span multiple artificial intelligence data centers (AIDCs). Two representative use cases illustrate the need for tighter integration between optical transport networks and compute orchestration systems.¶
In distributed training of large language models (LLMs), state-of-the-art techniques like 3D parallelism (data, pipeline, and tensor parallelism) coordinate thousands of GPUs across geographically dispersed AIDCs. The dominant synchronization primitive, Ring All-Reduce, is highly sensitive to the quality of inter-AIDC connections.¶
Consider a training job spanning Beijing and Shanghai (1000 km). The Round-Trip Time (RTT) is physically bound to roughly 10ms by the speed of light in fiber. However, standard RoCEv2 (RDMA over Converged Ethernet) networks rely on Priority Flow Control (PFC) to prevent packet loss. In a wide-area context, PFC can cause "head-of-line" blocking and deadlocks, leading to severe jitter and occasional large latency spikes. These unpredictable fluctuations cause the All-Reduce operation to stall. This results in massive GPU idle cycles and renders the training efficiency of GPUs, especially those using high-end H100 or Blackwell architectures.¶
Existing computing-aware routing frameworks (such as CATS) are designed for "steering" traffic over pre-existing paths and cannot eliminate this jitter. The core issue is the buffer-based queuing inherent to IP forwarding. To achieve deterministic performance, the optical physical layer is a superior choice.¶
An Optical Cross-Connect (OXC) can provision an ultra-low-latency, bufferless, and jitter-free wavelength circuit between two AIDCs. For a 1000km link, this provides a deterministic, hard-isolated channel with a fixed propagation delay (e.g., 5ms one-way) and zero queuing latency. This deterministic channel ensures that intra-iteration synchronization (All-Reduce) completes within a predictable time window, significantly improving GPU utilization and training stability.¶
Distributed AI inference, specifically for LLMs, introduces unique data dispersal requirements. The Key-Value (KV) Cache stores attention states for autoregressive generation. For long context windows and large batch sizes, the KV Cache can reach hundreds of gigabytes.¶
In a cross-regional inference scenario (e.g., an incoming request is processed by a central AIDC but must be offloaded to a regional edge node for sustained conversation), the KV Cache may need to be relocated or remotely accessed across the wide-area network.¶
If the KV Cache is migrated over a standard IP network, congestion and unpredictable buffer delays can significantly degrade the Time-to-First-Token (TTFT) and Time-Per-Output-Token (TPOT) metrics. For real-time conversational AI, even a single retransmission or queuing event can cause a noticeable stall or abrupt increase in latency, breaking the user experience. This is a critical bottleneck for implementing distributed inference services across data centers.¶
To enable seamless KV Cache migration, the transport network must provide a high-bandwidth, deterministic, and low-latency channel. An optical path (e.g., an fgOTN timeslot or OXC wavelength) can be dynamically provisioned to form a dedicated pipe specifically for the cache migration. This guarantees that the KV Cache transfer is not impacted by other data plane traffic, ensuring predictable TTFT and TPOT.¶
Furthermore, the optical path's guaranteed latency allows the inference scheduler to accurately predict the cost of remote KV Cache access or migration. This enables optimal placement decisions (e.g., co-locating the prompt processing and KV Cache affinity with the optical path quality).¶
Based on the problem statements and use cases described above, the following functional and performance requirements are necessary to support Optical Networks and AI Computing Orchestration (ONCO). These requirements are essential for enabling joint, high-performance scheduling of AI workloads across wide-area optical infrastructures.¶
An integrated control architecture is required to break down the management silos. This architecture must facilitate bidirectional fusion between the compute and network control planes.¶
Specifically, the architecture must support the following performance requirements:¶
A common language is needed for both domains to understand each other's capabilities and state. This requires a unified resource abstraction model.¶
The optical network's heterogeneous resources must be abstracted into a model that can be understood by the joint orchestrator. This model must expose detailed and quantifiable parameters including:¶
Similarly, heterogeneous compute resources (CPU, GPU, memory, storage) must be abstracted into a model that conveys their real-time capabilities and state to the joint orchestrator. This model must include:¶
This unified, detailed abstraction is the foundation for any credible joint decision-making process.¶
Building on the integrated architecture and unified abstraction, a joint orchestration mechanism is required to make end-to-end resource allocation decisions. This mechanism must:¶
This document makes no request for IANA action.¶
The following people contributed significantly to this document:¶