]> Zhongguancun Laboratory

Beijing China qinlc@mail.zgclab.edu.cn

Zhongguancun Laboratory

Beijing China suyy@mail.zgclab.edu.cn

Tsinghua University

Beijing China tolidan@tsinghua.edu.cn

Operations and Management BMWG RPKI, Relying Party, Benchmarking This document defines a benchmarking methodology for evaluating RPKI Relying Party (RP) implementations in controlled laboratory environments. The methodology focuses on whether RP implementations correctly perform required validation steps and on the performance of these operations. RP implementations are treated as black boxes, enabling consistent and objective assessment based on externally observable behavior rather than internal design or implementation details.

Introduction The Resource Public Key Infrastructure (RPKI) provides a framework for cryptographically securing Internet routing by allowing Relying Parties (RPs) to validate Route Origin Authorizations (ROAs) and other RPKI objects. Currently, there is no standardized methodology to evaluate whether RP implementations correctly perform the required validation procedures. In addition, the processing performance of RPs, such as the time required to validate objects and generate validated ROA payloads (VRPs), has not been systematically measured. This document defines a benchmarking methodology for Relying Parties that addresses both functional correctness and processing performance. Specifically, the methodology provides:

1. Functional correctness tests to evaluate compliance with the RFC requirements.
2. Performance tests to measure the total processing time from object retrieval to VRP generation.

This document currently focuses on the validation output produced from ROAs, namely the generated VRP set. Validation outputs associated with ASPA objects or other current or future RPKI signed objects are outside the scope of this document, although they may be evaluated using a similar methodology with object-specific validation requirements and expected outputs. The remainder of this document is structured as follows:

• Section 4 defines the functional correctness and performance tests.
• Section 5 defines the format for reporting test results.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Relying Party Overview A Relying Party (RP) in the Resource Public Key Infrastructure (RPKI) is responsible for retrieving, validating, and making available RPKI objects to support secure route validation. specifies the expected behavior of RPs, which includes:

• Object Retrieval: obtaining RPKI objects from Publication Points (PPs) and keeping them up-to-date.
• Object Syntax Validation: checking DER encoding, syntax, and structural correctness of RPKI objects, including certificates, CRLs, ROAs, and manifests.
• Certification Path Validation: constructing and validating certificate chains from the Trust Anchor to each leaf certificate.
• Signed Object Signature Validation: verifying digital signatures of RPKI objects.
• Manifest Processing: ensuring completeness and integrity of published objects.

After validation, the RP produces a Validated Payload for use in routing systems. These functions ensure that only valid and trusted RPKI objects influence routing decisions.

Test Setup This section defines the test setup. The System Under Test (SUT) (i.e., the RP software) is treated as a black box. No internal configuration or implementation behavior of the RP software is mandated. The test setup focuses on providing controlled inputs and observing RP outputs to enable reproducible and comparable measurements. In this methodology, the input to the SUT is the controlled RPKI repositories configured by the Controller and published through the controlled Servers. The output of the SUT is the generated VRP set, or equivalent validation output that allows the Controller to determine the resulting VRPs. Verification in this methodology is based on the SUT's externally observable output, rather than on its internal cache layout, internal processing pipeline, or implementation-specific debug information. For the correctness metrics defined in this document, a one-shot validation run is sufficient.

Test Environment In this methodology, the Tester consists of the Servers and the Controller. Together, they generate the test conditions, trigger events, and support observation of the SUT for functional and performance evaluation. The SUT itself is evaluated for correctness and efficiency in processing RPKI objects.

Test environment | Servers | +-------------+ +-------------+ ]]>

To ensure meaningful testing, the environment should include:

• At least one Trust Anchor (TA)
• One or more subordinate certificate authority (CA) certificates
• RPKI objects signed by CAs

The test environment should support multiple RPKI transport protocols for object retrieval, including Rsync , RRDP , or Erik .

System Under Test and Servers The SUT is an RPKI Relying Party implementation that retrieves RPKI objects from the controlled repositories, validates them, and generates the VRP set or equivalent validation output. The Servers should include PPs which host RPKI objects signed by CAs. Each PP at least contains ROAs, a manifest, and a CRL necessary for the tests. RPKI objects can be added, modified, or removed by the Controller.

Controller The Controller orchestrates the entire testing process. In , the arrows from the Controller to both the SUT and Servers represent the flow of control information. For the SUT, the Controller controls the start and end of tests. The Controller can obtain the SUT's VRP set or equivalent validation output through an externally observable output interface configured for the test, such as an output file, RTR, or a CCR profile . The Controller then processes test results and compares them with the expected output derived from the controlled repository contents. For the Servers, the Controller handles content changes to ensure that the SUT sees one set of RPKI objects during a validation run and a different set during the next run. Existing tools (e.g., ) can be used to implement and manage the test environment.

RPKI Relying Party Benchmarking Tests

Baseline Valid Repository Processing Objective: To establish a baseline for evaluating whether the SUT can process a controlled repository containing valid RPKI objects and generate the expected VRP set. This test verifies the SUT's output for a repository in which all relevant objects are valid and all validation conditions are satisfied. This baseline confirms that valid repository contents produce the expected VRP set before error cases are introduced. Procedure: The Servers and all RPKI objects are correctly configured prior to the test. The SUT is configured with the URI of the TA. The following steps are performed:

1. Publish a baseline repository containing valid certificates, manifests, CRLs, and ROAs on the Servers. The baseline repository MUST be constructed so that each ROA under test produces a distinct VRP payload. This ensures that the absence of a VRP from the generated VRP set can be attributed to the rejection of the corresponding ROA.
2. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
3. Start the SUT and allow it to perform one validation run over the controlled repository.
4. Verify that the generated VRP set matches the expected VRP set derived from the baseline repository contents.

Expected Results: The generated VRP set matches the expected VRP set derived from the baseline repository contents. This indicates that, for the baseline repository, the SUT can retrieve, parse, validate, and process the relevant RPKI objects sufficiently to generate the expected VRPs.

Object Syntax Validation Correctness This section evaluates whether the SUT correctly performs syntax validation for different RPKI objects. The SUT is expected to perform syntax checks according to the relevant specifications and detect objects that do not conform to the defined syntax requirements. The syntax requirements for different RPKI objects are defined in the following specifications:

• Certificate and CRL: and
• Manifest: and
• ROA: and

Unless otherwise specified, each test in this section starts from the baseline repository defined in the Baseline Valid Repository Processing test. A single object in the baseline repository is then replaced with a malformed object that violates one specific syntax requirement. The SUT behavior is evaluated by checking whether the generated VRP set matches the expected result derived from this single-object modification. In general, VRPs whose production depends on the malformed object are expected to be absent from the generated VRP set, while unaffected baseline VRPs are expected to remain present.

DER Decoding Objective: To evaluate whether the SUT correctly rejects an RPKI object that contains invalid Distinguished Encoding Rules (DER) encoding. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one ROA in the baseline repository whose payload is present in the baseline expected VRP set.
3. Replace only the selected ROA with a malformed DER-encoded version of that ROA, for example by using a BER encoding that is not valid DER.
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the generated VRP set matches the baseline expected VRP set with the VRP derived from the malformed ROA removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRP derived from the malformed DER-encoded ROA removed. Unaffected baseline VRPs remain present.

Certificate Syntax Validation Objective: To evaluate whether the SUT correctly performs syntax validation for CA certificates. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one leaf CA certificate in the baseline repository. The selected leaf CA has no subordinate CAs and issues EE certificates for one or more ROAs that produce baseline VRPs.
3. Replace only the selected leaf CA certificate with a malformed CA certificate that violates one specific syntax requirement defined in Section 7.2 of (e.g., missing mandatory fields, invalid field values, incorrect extensions, or malformed structures).
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs whose EE certificates are issued by the selected leaf CA removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs whose EE certificates are issued by the selected leaf CA removed. Unaffected baseline VRPs remain present.

CRL Syntax Validation Objective: To evaluate whether the SUT correctly performs syntax validation for CRLs. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one leaf CA in the baseline repository. The selected leaf CA has no subordinate CAs and its publication point contains one or more ROAs that produce baseline VRPs.
3. Replace only the CRL issued by the selected leaf CA with a malformed CRL that violates one specific syntax requirement defined in or .
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed. Unaffected baseline VRPs remain present.

Manifest Syntax Validation Objective: To evaluate whether the SUT correctly performs syntax validation for manifests. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one leaf CA in the baseline repository. The selected leaf CA has no subordinate CAs and its publication point contains one or more ROAs that produce baseline VRPs.
3. Replace only the manifest associated with the selected leaf CA's publication point with a malformed manifest that violates one specific syntax requirement defined in or .
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed. Unaffected baseline VRPs remain present.

ROA Syntax Validation Objective: To evaluate whether the SUT correctly performs syntax validation for ROAs. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one ROA in the baseline repository and record the VRP payload expected from that ROA.
3. Replace only the selected ROA with a malformed ROA that violates one specific syntax requirement defined in or .
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the VRP payload expected from the malformed ROA is absent from the generated VRP set.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRP payload expected from the malformed ROA removed. Unaffected baseline VRPs remain present.

Certification Path Validation Correctness Objective: To evaluate whether the SUT correctly performs certification path validation for the EE certificate associated with a selected ROA. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one ROA in the baseline repository and identify the EE certificate associated with that ROA.
3. Modify only the relationship between the selected ROA's EE certificate and its issuing CA certificate to introduce one certification path violation. Examples of such violations include, but are not limited to:
   • Invalid certificate signature, where the EE certificate cannot be verified using the issuing CA's public key.
   • Resource extension violation, where the resources listed in the EE certificate are not encompassed by the resources listed in the issuing CA certificate.
   • Invalid issuer-subject relationship, where the issuer information in the EE certificate is inconsistent with the issuing CA certificate.
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the VRP payload expected from the selected ROA is absent from the generated VRP set.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRP payload expected from the selected ROA removed. Unaffected baseline VRPs remain present.

Signed Object Signature Validation Correctness Objective: To evaluate whether the SUT correctly verifies the digital signature of an RPKI signed object using the corresponding public key in the associated certificate. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one RPKI signed object in the baseline repository whose validity is required for producing one or more expected VRPs. For a ROA signature test, select one ROA and record the VRP payload expected from that ROA.
3. Replace only the selected signed object with a version whose signature is intentionally invalid (e.g., the object content is modified after signing or the signature does not match the corresponding certificate).
4. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
5. Start the SUT and allow it to synchronize the repository contents.
6. Verify that the generated VRP set matches the baseline expected VRP set with the VRPs that depend on the invalidly signed object removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRPs whose production depends on the invalidly signed object removed. For a ROA signature test, this means that the VRP payload expected from the selected ROA is removed. Unaffected baseline VRPs remain present.

Manifest Processing Correctness This section evaluates whether the SUT correctly uses manifests to verify the integrity and completeness of RPKI repository objects. Manifests are expected to be used to:

• Verify that the content of each RPKI object matches the hash listed in the manifest.
• Verify that objects accepted from a publication point are consistent with the object list declared in the corresponding manifest.

Manifest Hash-Mismatch Check Objective: To evaluate whether the SUT correctly handles a retrieved RPKI object whose content does not match the hash declared in its associated manifest. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one leaf CA in the baseline repository. The selected leaf CA has no subordinate CAs and its publication point contains one or more ROAs that produce baseline VRPs.
3. Select one ROA listed in the manifest associated with the selected leaf CA's publication point.
4. Modify only the selected ROA after the manifest has been generated, so that the ROA hash no longer matches the hash listed in the manifest.
5. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
6. Start the SUT and allow it to synchronize the repository contents.
7. Verify that the generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed.

Expected Results: The generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed. Unaffected baseline VRPs from other publication points remain present. This reflects that, in a cold-start validation run, a hash mismatch causes the fetch for that publication point to fail.

Manifest Object-Mismatch Check Objective: To evaluate whether the SUT correctly handles an object that is missing from or extra in relation to the object list declared in a manifest. Procedure:

1. Start from the baseline repository defined in the Baseline Valid Repository Processing test.
2. Select one leaf CA in the baseline repository. The selected leaf CA has no subordinate CAs and its publication point contains one or more ROAs that produce baseline VRPs.
3. For a missing-object test case, remove only one object listed in the manifest from the selected leaf CA's publication point.
4. For an extra-object test case, add one otherwise valid ROA to the selected leaf CA's publication point without listing it in the corresponding manifest, and record the VRP payload that would be expected from that ROA if it were accepted.
5. Each test case introduces only one manifest object mismatch at a time.
6. Initialize the SUT for a cold-start validation run (e.g., with no previously cached repository state).
7. Start the SUT and allow it to synchronize the repository contents.
8. Verify that the generated VRP set matches the expected result for the specific object-mismatch test case.

Expected Results: For a missing-object test case, the generated VRP set matches the baseline expected VRP set with the VRPs derived from ROAs in the selected leaf CA's publication point removed. Unaffected baseline VRPs from other publication points remain present. This reflects that, in a cold-start validation run, a missing object listed in the manifest causes the fetch for that publication point to fail. For an extra-object test case, the generated VRP set matches the baseline expected VRP set. The VRP payload that would be derived from the extra ROA is absent from the generated VRP set.

Processing Time test Objective: To measure the end-to-end processing time of the SUT for a one-shot validation run. Processing time is measured as the elapsed wall-clock time between the moment the Controller starts the validation run and the moment the SUT makes the generated VRP set or equivalent validation output available. Procedure:

1. Prepare a test repository containing a predefined set of RPKI objects.
2. Start the SUT for a one-shot validation run and record the start timestamp.
3. Allow the SUT to complete the validation run and produce the generated VRP set or equivalent validation output.
4. Record the end timestamp when the generated VRP set or equivalent validation output becomes available.
5. Calculate the total processing time as the difference between the start and end timestamps.
6. Repeat the test for different numbers of CAs, ROAs, and generated VRPs to evaluate performance under varying load conditions.

Report Format This section defines the format for reporting the results of RPKI Relying Party benchmarking tests. The format is concise and suitable for documenting both functional correctness and performance results. System Under Test (SUT) Information:

• RP implementation name and version
• Relevant SUT configuration used for the test

Test Repository Information:

• Number of objects and object types used in the baseline repository
• Number of expected VRPs in the baseline repository
• Test cases executed
• Test-specific repository modification, if any

Functional Correctness Test Results:

• Baseline Valid Repository Processing: pass/fail
• DER Decoding Correctness:
  • Invalid DER Handling: pass/fail
• Object Syntax Validation Correctness:
  • Certificate: pass/fail
  • CRL: pass/fail
  • Manifest: pass/fail
  • ROA: pass/fail
• Certification Path Validation Correctness: pass/fail
• Signed Object Signature Validation Correctness:
  • Invalid Signature Handling: pass/fail
• Manifest Processing Correctness:
  • Hash-Mismatch Check: pass/fail
  • Object-Mismatch Check:
    • Missing Object: pass/fail
    • Extra Object: pass/fail

Performance Test Results:

• Processing Time Performance Test:
  • Number of CAs
  • Number of ROAs
  • Number of generated VRPs
  • Total elapsed time from the start of the one-shot validation run to the availability of the generated VRP set

Notes:

• Functional correctness tests report pass/fail based on whether the SUT's output matches the expected output derived from the controlled repository contents.
• Performance tests report total processing time together with the test scale parameters defined above.

Security Considerations This document defines benchmarking methodologies for RPKI RP implementations in controlled laboratory environments using dedicated address space and constrained resources. No additional security considerations are identified within the scope of this document.

IANA Considerations This document has no IANA requests.

Acknowledgements The authors would like to thank Jorge Cano and the FORT team for their detailed reviews and constructive feedback, which helped clarify and improve the scope and methodology of this work. The authors also thank Mikhail Puzanov for his valuable feedback on the RP/SUT model and the externally observable validation output used in this methodology.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References This document describes an architecture for an infrastructure to support improved security of Internet routing. The foundation of this architecture is a Resource Public Key Infrastructure (RPKI) that represents the allocation hierarchy of IP address space and Autonomous System (AS) numbers; and a distributed repository system for storing and disseminating the data objects that comprise the RPKI, as well as other signed objects necessary for improved routing security. As an initial application of this architecture, the document describes how a legitimate holder of IP address space can explicitly and verifiably authorize one or more ASes to originate routes to that address space. Such verifiable authorizations could be used, for example, to more securely construct BGP route filters. This document is not an Internet Standards Track specification; it is published for informational purposes. This document defines a "manifest" for use in the Resource Public Key Infrastructure (RPKI). A manifest is a signed object (file) that contains a listing of all the signed objects (files) in the repository publication point (directory) associated with an authority responsible for publishing in the repository. For each certificate, Certificate Revocation List (CRL), or other type of signed objects issued by the authority that are published at this repository publication point, the manifest contains both the name of the file containing the object and a hash of the file content. Manifests are intended to enable a relying party (RP) to detect certain forms of attacks against a repository. Specifically, if an RP checks a manifest's contents against the signed objects retrieved from a repository publication point, then the RP can detect replay attacks, and unauthorized in-flight modification or deletion of signed objects. This document obsoletes RFC 6486. This document defines a standard profile for X.509 certificates for the purpose of supporting validation of assertions of "right-of-use" of Internet Number Resources (INRs). The certificates issued under this profile are used to convey the issuer's authorization of the subject to be regarded as the current holder of a "right-of-use" of the INRs that are described in the certificate. This document contains the normative specification of Certificate and Certificate Revocation List (CRL) syntax in the Resource Public Key Infrastructure (RPKI). This document also specifies profiles for the format of certificate requests and specifies the Relying Party RPKI certificate path validation procedure. [STANDARDS-TRACK] This document defines a generic profile for signed objects used in the Resource Public Key Infrastructure (RPKI). These RPKI signed objects make use of Cryptographic Message Syntax (CMS) as a standard encapsulation format. [STANDARDS-TRACK] In the Resource Public Key Infrastructure (RPKI), Certificate Authorities (CAs) publish certificates, including end-entity certificates, Certificate Revocation Lists (CRLs), and RPKI signed objects to repositories. Relying Parties retrieve the published information from those repositories. This document specifies a new RPKI Repository Delta Protocol (RRDP) for this purpose. RRDP was specifically designed for scaling. It relies on an Update Notification File which lists the current Snapshot and Delta Files that can be retrieved using HTTPS (HTTP over Transport Layer Security (TLS)), and it enables the use of Content Distribution Networks (CDNs) or other caching infrastructures for the retrieval of these files. This document provides a single reference point for requirements for Relying Party (RP) software for use in the Resource Public Key Infrastructure (RPKI). It cites requirements that appear in several RPKI RFCs, making it easier for implementers to become aware of these requirements. Over time, this RFC will be updated to reflect changes to the requirements and guidance specified in the RFCs discussed herein. This document defines a standard profile for Route Origin Authorizations (ROAs). A ROA is a digitally signed object that provides a means of verifying that an IP address block holder has authorized an Autonomous System (AS) to originate routes to one or more prefixes within the address block. This document obsoletes RFC 6482. This memo profiles the X.509 v3 certificate and X.509 v2 certificate revocation list (CRL) for use in the Internet. An overview of this approach and model is provided as an introduction. The X.509 v3 certificate format is described in detail, with additional information regarding the format and semantics of Internet name forms. Standard certificate extensions are described and two Internet-specific extensions are defined. A set of required certificate extensions is specified. The X.509 v2 CRL format is described in detail along with standard and Internet-specific extensions. An algorithm for X.509 certification path validation is described. An ASN.1 module and examples are provided in the appendices. [STANDARDS-TRACK] BSD Software Development RIPE NCC APNIC JPNIC This document specifies the Erik Synchronization Protocol for use with the Resource Public Key Infrastructure (RPKI). Erik Synchronization can be characterized as a data replication system using Merkle trees, a content-addressable naming scheme, concurrency control using monotonically increasing sequence numbers, and HTTP transport. Relying Parties can combine information retrieved via Erik Synchronization with other RPKI transport protocols. The protocol's design is intended to be efficient, fast, easy to implement, and robust in the face of partitions or faults in the network. BSD Software Development RIPE NCC RIPE NCC OpenBSD This document specifies a Canonical Cache Representation (CCR) content type for use with the Resource Public Key Infrastructure (RPKI). CCR is a Distinguished Encoding Rules (DER) encoded data interchange format which can be used to represent various aspects of the state of a validated RPKI cache at a particular point in time. The CCR profile is a compact and versatile format, well-suited for a variety of applications, for example, audit trails, analytics pipelines, and validated payload dissemination. n.d. n.d.