| Internet-Draft | IOAM Using MNA | May 2025 |
| Gandhi, et al. | Expires 23 November 2025 | [Page] |
In situ Operations, Administration, and Maintenance (IOAM), defined in RFC 9197, is an on-path telemetry method to collect and record the operational state and telemetry information using, for example, Pre-allocated, Proof-of-Transit, Edge-To-Edge or Incremental IOAM Option, that can be used to calculate various performance metrics. RFC 9326 defined the IOAM Direct Export (IOAM-DEX) Option in which the operational state and telemetry information are collected according to the specified profile and exported in a manner and format defined by a local policy on each node along the path.¶
MPLS Network Actions (MNA) techniques are meant to indicate actions to be performed on any combination of Label Switched Paths, MPLS packets, and the node itself, and to transfer data needed for these actions. This document explores the MNA mechanisms to collect and transport the on-path operational state, and telemetry information IOAM data fields, including IOAM-DEX Option.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 23 November 2025.¶
Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
In situ OAM (IOAM) [RFC9197] is an on-path telemetry method used to collect and record the operational state and telemetry information that can be used to calculate various performance metrics. Several IOAM Option types (e.g., Pre-allocated, Proof-of-Transit, Edge-To-Edge, and Incremental) use the user packet to collect the operational state and telemetry information. Such mechanisms transport the collected information to an IOAM decapsulating node (typically located at the edge of the IOAM domain within the data packet). IOAM Direct Export (IOAM-DEX) [RFC9326] is an IOAM Option type. In IOAM-DEX, the operational state and telemetry information are collected according to the specified profile and exported in a manner and format defined by a local policy on each node along the path.¶
MPLS Network Actions (MNA) techniques [I-D.ietf-mpls-mna-fwk] indicate actions to be performed on any combination of Label Switched Paths, MPLS packets, the node itself, and also allow for the transfer of data needed for these actions. [I-D.ietf-mpls-mna-hdr] defines mechanisms for carrying the Network Action Sub-Stack (NAS) as part of the MPLS label stack, i.e., In-Stack Data (ISD) MNA solution. [I-D.ietf-mpls-mna-ps-hdr] defines mechanisms for carrying MNA and Ancillary Data (AD) outside the MPLS label stack, i.e., as the Post-Stack Data (PSD) MNA solution. [I-D.ietf-mpls-mna-usecases] describes various use cases that can be realized using MNA techniques, including IOAM Pre-allocated, Proof-of-Transit, Edge-To-Edge, Incremental, and IOAM-DEX Option types.¶
This document describes how MNA can be used for collecting and transporting on-path operational state and telemetry information using IOAM data fields for IOAM Option types, including IOAM-DEX. Specifying the mechanism for exporting the collected information in case of the IOAM-DEX Option is outside the scope of this document.¶
OAM: Operations, Administration, and Maintenance¶
HBH: Hop-By-Hop¶
I2E: Ingress-To-Egress¶
IHS: Ingress-To-Egress (I2E), Hop-By-Hop (HBH) or Select Scope¶
IOAM: In situ OAM¶
IOAM-DEX: IOAM Direct Export¶
IOAM-DEX-ISD-MNA: IOAM Direct Export as In-Stack Data MPLS Network Action¶
ISD: In-Stack Data¶
PSD: Post-Stack Data¶
PSH: Post-Stack Header¶
LSE: Label Stack Entry¶
MPLS: Multiprotocol Label Switching¶
MNA: MPLS Network Action¶
NAI: Network Action Indicator¶
NAS: Network Action Sub-stack¶
NASL: Network Action Sub-stack Length¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
Pre-allocated, Incremental, Proof-of-Transit, and Edge-to-Edge IOAM Option types [RFC9197] use user packets to collect and transport the operational state and telemetry information. In some environments, for example, data center networks, this technique is useful as the available bandwidth and the use of jumbo frames can accommodate the increase of the packet payload. This document defines the PSD MNA-based ([I-D.ietf-mpls-mna-ps-hdr]) solution supporting Pre-allocated, Incremental, Proof-of-Transit, and Edge-to-Edge IOAM Option types (Section 4.1).¶
However, for some use cases, e.g., mobile backhaul, in which network resources are closely controlled, collecting and transporting the telemetry information within a data packet may noticeably decrease the cost-efficiency of network operations. Although the operational state and telemetry information are essential for network automation (Section 4 of [RFC8969]), its delivery is not as critical as user packets. As such, collecting and transporting the operational state and telemetry information using the management plane is a viable option for some environments. IOAM-DEX [RFC9326] is capable of collecting all IOAM data fields defined in [RFC9197]. The processing and transport of the collected information are controlled by a local policy which is outside the scope of this specification. The performance considerations discussed in Section 5 of [RFC9326] are applicable here. In this document, the realizations of IOAM-DEX using the ISD MNA approach ([I-D.ietf-mpls-mna-hdr] and PSD MNA ([I-D.ietf-mpls-mna-ps-hdr]) are defined in Section 4.2 and Section 4.1, respectively.¶
The presence of a PSD MNA is indicated in an In-Stack Network Action Sub-stack is defined in [I-D.ietf-mpls-mna-ps-hdr] and is shown in Figure 1.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MNA Label | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Opcode | Data (Format B) |P|IHS|S|U| NASL | NAL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Opcode | Data (Format C) |S|U| Data | NAL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label | TC |1| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Post-Stack MNA for IOAM as shown in Figure 2 ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| ~ Optional Payload + Padding ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The MNA Label (value TBA4) is defined in [I-D.ietf-mpls-mna-hdr].¶
The IHS scope field defined in [I-D.ietf-mpls-mna-hdr] is used to indicate that I2E or HBH or Select processing is required for the Network Action and Ancillary Data.¶
If both edge and intermediate nodes need to process the IOAM data fields then IHS scope MUST be set to "HBH, value 0x1". If only edge nodes need to process the IOAM data fields then IHS scope MUST be set to "I2E, value 0x0". The HBH scope allows to skip the IOAM data processing on the intermediate nodes i.e., avoids the need to parse all IOAM data fields to detect the HBH option type.¶
The U Flag for Unknown Action Handling is specified in [I-D.ietf-mpls-mna-hdr].¶
S bit is the Bottom of Stack field [RFC3032].¶
IHS field is set as specified in [I-D.ietf-mpls-mna-hdr].¶
P flag is set as specified in [I-D.ietf-mpls-mna-ps-hdr].¶
The Network Action Sub-Stack Length (NASL) is set as specified in [I-D.ietf-mpls-mna-hdr].¶
Opcode (7 bits) value is set to TBA1 (for In-Stack Network Action for IOAM and IOAM-DEX in PSD) for carrying an IOAM Option-Type and the associated data fields defined in [RFC9197] and the IOAM-DEX Option-Type and the associated data fields defined in [RFC9326] in Post-Stack Data MNA.¶
The In-Stack Network Action with IOAM and IOAM-DEX in PSD is optional and can be carried in Format B or Format C Label Stack Entry (LSE).¶
Data (10 bits, next to the Opcode field) in LSE contains the offset for PSD for this In-Stack Network Action in 4-octets units after BOS LSE to the start of the Post-Stack Data MPLS Network Action Opcode. Due to the Post-Stack Header, minimum value for the offset is 1 (i.e, 4-octets).¶
Length of Network Action (NAL) [I-D.ietf-mpls-mna-hdr] is set to 0 for the In-Stack Network Action for IOAM and IOAM-DEX in PSD.¶
A packet may carry more than one In-Stack Network Action for IOAM and IOAM-DEX in PSD in an MNA Sub-Stack (for example, for different IOAM Option-Types as identified in Post-Stack Data MNA). There may be a different In-Stack Network Action (other than for the IOAM and IOAM-DEX) in the In-Stack MNA Sub-Stack.¶
An example encoding for PSD MNA carrying IOAM is shown in Figure 2.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |N N N N|Version| PS-HDR-LEN | TYPE = POST-STACK-HDR-MNA = 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| |PS-NA-OPCODE |R|R| PS-NAL |R|R| BLOCK-NUMBER|IOAM-OPT-TYPE| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ IOAM Option and Data Space [RFC9197] [RFC9326] ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
The Post-Stack Header (PSH) is added with the following fields as defined in [I-D.ietf-mpls-mna-ps-hdr].¶
The Post-Stack Data MNA for IOAM encoding contains Post-Stack Network Action Opcode for IOAM, length in number of 4-octet units, and IOAM Option-Type with IOAM data fields in ancillary data as shown in Figure 2. The IOAM data fields MUST follow the definitions corresponding to their IOAM-Option-Types (e.g., see Section 4.4 of [RFC9197] and Section 3 of [RFC9326]).¶
An IOAM option is added in the Post-Stack Data MNA containing the following fields:¶
An example of multiple PSD MNAs carrying different IOAM option types is shown in Figure 3.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |N N N N|Version| PS-HDR-LEN | TYPE = POST-STACK-HDR-MNA = 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| |PS-NA-OPCODE |R|R| PS-NAL |R|R| BLOCK-NUMBER|IOAM-OPT-TYPE| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ IOAM Option and Data Space [RFC9197] [RFC9326] ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |PS-NA-OPCODE |R|R| PS-NAL |R|R| BLOCK-NUMBER|IOAM-OPT-TYPE| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ IOAM Option and Data Space [RFC9197] [RFC9326] ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Optional Payload + Padding ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This document uses all the elements of the IOAM-DEX Option-Type format defined in [RFC9326] to support IOAM-DEX in an MPLS network using MPLS Network Action (MNA) as in-stack data (ISD) [I-D.ietf-mpls-mna-hdr].¶
To support the direct export of the operational state and telemetry information, the IOAM-DEX-ISD-MNA blob (binary large object) is placed as part of the ISD block in an MPLS label stack according to the MNA encoding principles defined in [I-D.ietf-mpls-mna-hdr].¶
Using the IHS field, the IOAM-DEX-ISD-MNA can be performed in Hop-by-Hop, Ingress-to-Egress, or Select modes [I-D.ietf-mpls-mna-fwk] of collecting the operational state and telemetry information, using an MNA Opcode (Figure 4).¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MNA Label | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Opcode = TBA2 | Data |P|IHS|S|U| NASL | NAL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~1| IOAM-DEX-ISD-MNA |S| ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Here, the enclosed elements are defined as follows:¶
Policies controlling the processing of the collected operational state and telemetry information, and its transport are outside the scope of this document.¶
The IOAM-DEX in MNA header uses the LSE Format D, as defined in Section 4.4 [I-D.ietf-mpls-mna-hdr], that maps IOAM-DEX Option Type format [RFC9326]. In addition to the requirement to preserve the Bottom of Stack field, the most significant bit in LSE Format D is always set to 1 avoiding a possible mix-up of the LSE with one of the Base Special Purpose Labels. The format of IOAM-DEX-MNA blob in In-Stack MNA header is shown in Figure 5.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| Namespace-ID | Resv |S| Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| IOAM-Trace-Type-MNA |S|O|R| Ext-Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| Flow ID MNA (Optional) |S| Flow ID MNA | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| Sequence Number MNA (Optional) |S| Seq Num MNA | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Where fields are defined as follows:¶
0 1 2 3 4 5
+-+-+-+-+-+-+
|F|S|U|U|U|U|
+-+-+-+-+-+-+
Where fields are defined as follows:¶
The I2E IOAM data fields carry the IOAM Option-Type(s) that require processing on the encapsulating and decapsulating nodes only.¶
The IOAM Option-Type carried can be IOAM Edge-To-Edge Option-Type (value 3) defined in [RFC9197] as well as DEX Option-Type (value 4) defined in [RFC9326]. The I2E IOAM data fields SHOULD NOT carry any IOAM Option-Type that require IOAM processing on the intermediate nodes as it will not be processed by them when IHS scope is set to "I2E, value 0x0".¶
The I2E IOAM and IOAM-DEX Network Action procedure is summarized as following:¶
The HBH IOAM data fields carry the Option-Type(s) that require processing at the intermediate and/or encapsulating and decapsulating nodes.¶
The IOAM Option-Type carried can be IOAM Pre-allocated Trace Option-Type (value 0), IOAM Incremental Trace Option-Type (value 1) and IOAM Proof of Transit (POT) Option-Type (value 2), and Edge-To-Edge Option-Type (value 3) defined in [RFC9197] as well as DEX Option-Type (value 4) defined in [RFC9326].¶
Editor's note: IPv6 option is not supported for HBH IOAM Incremental Trace Option-Type (value 1). Similarly, MPLS network action is also not supported for HBH IOAM Incremental Trace Option-Type (value 1) and will be removed.¶
The Hop-By-Hop IOAM and IOAM-DEX Network Action procedure is summarized as following:¶
Both HBH and I2E Scope IOAM may be carried in PSD MNA in an MPLS packet. In this case, the PSD MNA with HBH IOAM data fields MUST be added after the BOS and before the PSD MNA with I2E IOAM data fields. This way, the RLD required to process them on the intermediate nodes is minimized.¶
The decapsulating node that needs to remove the IOAM and IOAM-DEX data fields and perform the IOAM and IOAM-DEX functions may not be capable of supporting it. The encapsulating node needs to know if the decapsulating node can support the IOAM and IOAM-DEX functions. The signaling extension for this capability exchange is outside the scope of this document.¶
The intermediate node that is not capable of supporting the IOAM and IOAM-DEX functions defined in this document, can simply skip the IOAM and IOAM-DEX processing.¶
When a packet is received with MPLS Encapsulated Network Action for IOAM and IOAM-DEX, the nested MPLS encapsulating node that needs to add different Network Action for IOAM and IOAM-DEX, the node MUST add a new MNA Sub-Stack with the Network Action for IOAM and IOAM-DEX as part of the new MPLS encapsulation.¶
The encapsulating node needs to make sure that the IOAM and IOAM-DEX data fields in MNA are added within the Readable Label Depth (RLD) of the downstream MNA capable nodes in order for them to be able to process the IOAM and IOAM-DEX.¶
Security considerations discussed in [RFC9197], [RFC9326], [I-D.ietf-mpls-mna-hdr], [I-D.ietf-mpls-mna-ps-hdr] and [I-D.ietf-mpls-mna-fwk] apply to this document.¶
The usage of MPLS network actions defined in this document for IOAM and IOAM-DEX is intended for deployment in a single network administrative domain. As such, it assumes that the operator enabling the IOAM and IOAM-DEX operations has previously verified the integrity of the path. Still, operators need to properly secure the IOAM and IOAM-DEX in the domain to avoid malicious configuration and use, which could include injecting malicious IOAM and IOAM-DEX packets into the domain.¶
IANA is requested to assign codepoints from its Network Action Opcodes registry (creation requested in [I-D.ietf-mpls-mna-hdr]) as specified in Table 1.¶
| Opcode | Description | Reference |
|---|---|---|
| TBA1 | In-Stack Network Action for IOAM and IOAM-DEX in PSD | This document |
| TBA2 | In-Stack Network Action for IOAM-DEX in ISD | This document |
IANA is requested to assign a codepoint for the IOAM and IOAM-DEX Network Action Opcode from the Post-Stack Data MPLS Network Action Opcode registry (to be created by in [I-D.ietf-mpls-mna-ps-hdr]).¶
| Value | Description | Reference |
|---|---|---|
| TBA3 | Post-Stack Data MPLS Network Action for IOAM | This document |
The authors would like to thank Patrick Khordoc, Sagar Soni, Shwetha Bhandari, Vengada Prasad Govindan, Tarek Saad, Stewart Bryant, Xiao Min, Jaganbabu Rajamanickam, and Cheng Li for providing many useful comments. The authors would also like to thank Mach Chen, Andrew Malis, Matthew Bocci, and Nick Delregno for the MPLS-RT expert review of the early version of this document.¶
The following people have substantially contributed to this document:¶
Zafar Ali Cisco Systems, Inc. Email: zali@cisco.com Loa Andersson Huawei Technologies Email: loa@pi.nu Frank Brockners Cisco Systems, Inc. Hansaallee 249, 3rd Floor DUESSELDORF, NORDRHEIN-WESTFALEN 40549 Germany Email: fbrockne@cisco.com Voitek Kozak Comcast Email: Voitek_Kozak@comcast.com¶