Internet-Draft NRTM v4 February 2022
Romijn, et al. Expires 7 August 2022 [Page]
Workgroup:
GROW
Internet-Draft:
draft-spaghetti-grow-nrtm-v4-00
Published:
Intended Status:
Standards Track
Expires:
Authors:
S. Romijn
Reliably Coded
J. Snijders
Fastly
E. Shryane
RIPE NCC
S. Konstantaras
AMS-IX

Near Real Time Mirroring (NRTM) version 4

Abstract

This document specifies a one-way synchronization protocol for Internet Routing Registry (IRR) records. The protocol allows instances of IRR database servers to mirror IRR records, specified in in the Routing Policy Specification Language (RPSL), between each other.

Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 7 August 2022.

Table of Contents

1. Introduction

The Internet Routing Registry (IRR) consists of several different IRR Databases, each storing objects in the Routing Policy Specification Language (RPSL). About a dozen larger IRR Databases are well known and widely used, operated by different organisations, like RIRs and some large network operators. IRR objects serve many purposes, ranging from manual research by operators to automated network configuration and filtering.

Most of these well known IRR Databases mirror IRR objects from some others, so that queries run against these instances provide a comprehensive view. Some parties also mirror IRR Databases to private IRR server instances, to reduce latency in queries, analyze IRR objects, or other purposes.

NRTM version 4 is a protocol for IRR mirroring, designed to address issues in existing IRR Database mirroring protocols. In NRTMv4, IRR Databases publish their records on a HTTPS endpoint, with periodic Snapshot Files and regular Delta Files. Signing allows integrity checks. By only generating files once and publishing them over HTTPS, scalability is dramatically improved. It borrows some concepts in [RFC8182], as there are overlaps between the two protocols.

2. Informal overview

In NRTMv4, a mirror server is an instance of IRR Database software that has a database of IRR objects and publishes them to allow mirroring by others. This can be retrieved by mirror clients, which then load the IRR objects into their local storage.

Publication consists of three different files:

Mirror clients initially retrieve the small Update Notification File and a Snapshot File, from which they initialize their local copy of the Database. After that, mirror clients only retrieve the Update Notification File periodically to determine whether there are any changes, and then retrieve only the relevant Delta Files, if any. This minimizes data transfer. Deltas have sequential versions.

A mirror client is configured from a Database Configuration File on a well-known URL, which contains a public signing key which it uses to verify the Update Notification File, which in turn contains hashes of all the Snapshot and Delta Files.

Upon initialization, the mirror server generates a session ID for the Database. This allows long term caching and used by the client to determine that the Delta Files continue to form a full set of changes allowing an update to the latest version. If the mirror server loses partial history, or the mirror client starts mirroring from a different server, the session ID change will force a full reload from the latest Snapshot File, ensuring there are no accidental mirroring gaps.

Mirror servers can use caching to reduce their load, particularly because snapshots and deltas are immutable for a given session ID and version number. These are also the largest files. Update Notification Files may not be cached for longer than one minute, but are fairly small.

Note that in NRTMv4, a contiguous version number is used for the Database version and Delta Files. This is different and unrelated to the serial in NRTMv3. NRTMv3 serials refer to a single change to a single object, whereas a NRTMv4 version refers to one delta, possibly containing multiple changes to multiple objects. NRTMv3 serials can also contain gaps, NRTMv4 versions may not.

3. Mirror server use

3.1. Key Configuration and the Database Configuration File

REMOVE BEFORE PUBLICATION: this whole section is wide open to suggestions / ideas

When enabling NRTMv4 publication for an IRR Database, the operator MUST generate and configure a private Ed25519 [RFC8032] key. Then, the operator MUST publish the public key, the name of the IRR Database and the URL of the Update Notification File in the Database Configuration File. This is a JSON file on a well-known URI on the "irr-nrtm.json" suffix on the operator's authoritative domain name [RFC8615].

The authoritative domain name is the domain which is generally known by the IRR community to belong to the operator of the IRR Database. Note that the Update Notification File, Snapshot File and Delta Files MAY also be hosted on the authoritative domain, or an entirely different host or domain.

It is RECOMMENDED that implementations provide easily accessible tools for operators to generate new Ed25519 keys to enter into their configuration, and propose a Database Configuration File from the current configuration. Configuration options SHOULD be clearly named to indicate that they are private keys.

3.2. Snapshot Initialization

A mirror server MUST follow the initialization steps upon the first export for an IRR Database by that mirror server, or if the server lost history and can not reliably produce a continuous set of deltas from a previous state.

In other words, either the mirror server guarantees that clients following the deltas have a correct and complete view, or MUST reinitialize, which will force clients to reinitialize as well.

Initialization consists of these actions:

Note that session IDs, versions and all files always relate to a specific IRR Database. For example, a mirror server publishing NRTMv4 for RIPE and RIPE-NONAUTH, will generate two Update Notification Files, referring two Snapshot Files, and two sets of Delta Files each with contiguous version numbers - all completely independent to each other, with different session IDs. This applies even if the same IRR server instance produces both.

3.3. Publishing updates

3.3.1. Delta Files

Changes to IRR objects MUST be recorded in Delta Files. One Delta File can contain multiple changes.

Updates are generated as follows:

  • A mirror server MUST publish a Delta File approximately every minute, if there have been changes to IRR objects in that time frame.
  • If multiple changes have occurred within the time frame that would cancel each other out, like an addition and immediate deletion of the same object, the mirror server MUST still include all these changes.
  • If a mirror server is lagging in production of Delta Files, such as after an initialization or server downtime, it MUST generate one larger "catch up" Delta File, rather than individual Delta Files for every one minute window.
  • A new Delta File MUST be generated with a new version, one greater than the last Delta File version, or one greater than the last Snapshot File version if there were no prior deltas at all.
  • The Delta File MUST include all changes that happened during the time frame, in the order in which they occurred.
  • The URL where the Delta File is published MUST contain the session ID and version number to allow it to be indefinitely cached. It MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.
  • After generating a new Delta File, a mirror server MUST remove all Delta Files older than 24 hours.
  • The Update Notification File MUST be updated to include the new Delta File and update the database version.

3.3.2. Snapshot Files

Snapshot Files after initialization are generated as follows:

  • The mirror server MUST generate a new Snapshot File between once per hour and once per day, if there have been changes to the IRR objects.
  • The version number of the new snapshot MUST be equal to the last Delta File version.
  • If there have been no changes to the IRR objects since the last snapshot, the mirror server MUST NOT generate a new snapshot.
  • The URL where the Snapshot File is published MUST contain the session ID and version number to allow it to be indefinitely cached. It MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.
  • The Update Notification File MUST be updated to include the new snapshot, if one was generated.

3.3.3. Update Notification File

The Update Notification File must be updated when a new Delta or Snapshot File is published and, even if there have been no changes, at least every 24 hours.

After any update to the Update Notification file, the mirror server MUST also update the Update Notification Signature File for the new Update Notification File contents.

3.3.4. Publication Policy Restrictions

A mirror server MAY have a policy that restricts the publication of certain IRR objects or attributes, or modifies these before publication. Typical scenarios for this include preventing the distribution of certain personal data or password hashes, or excluding objects which do not meet validation rules like RPKI consistency. It is RECOMMENDED to modify objects in such a way that this change is evident to humans reading the object text, for example by adding remark lines or comments.

Mirror servers are RECOMMENDED to remove password hashes from the auth lines in mntner objects, as they have little use beyond the authoritative server, and their publication may be a security risk.

If a mirror server has a policy that restricts or modifies object publication, this MUST be applied consistently to Snapshot Files and Delta Files from the moment the policy is enacted or modified.

4. Mirror client use

4.1. Configuration

REMOVE BEFORE PUBLICATION: this whole section is wide open to suggestions / ideas

Mirror clients are configured with the name of an IRR Database to mirror and the domain considered authoritative by the mirror client operator, which hosts the Database Configuration File. The mirror client MUST retrieve this file, look for an entry under the expected IRR Database name, and read the Update Notification File URL and the public key. The mirror client MUST only allow retrieval of this file over HTTPS.

The mirror client MUST retrieve the Database Configuration File every 24 hours, and update its local configuration if any settings in the file have changed.

4.2. Initialization from snapshot

Clients MUST initialize from a Snapshot File when initially configured or if they are not able to update their local data from the provided Delta Files:

4.3. Processing Delta Files

If a mirror client has previously initialized from a snapshot:

If the Update Notification File or one of the Delta Files is rejected, the mirror client MUST NOT process any newer Deltas than those that are valid and have been successfully verified.

4.4. Signature and Staleness Verification

Every time a mirror client retrieves a new version of the Update Notification File, it MUST retrieve and verify the Update Notification Signature File. The signature MUST be valid for the configured public key for the contents of the Update Notification File.

If the signature does not match, the mirror client MUST reject the Update Notification File and retrieve the Database Configuration File, as a mismatch may be caused by a key rotation by the mirror server operator. If the Database Configuration File contains a new public key, the mirror client will attempt to use that key for signature verification instead.

A mirror client can use the generation timestamp in the Update Notification File to check whether the file is stale, as the mirror server must update this file at least every 24 hours. If the generation timestamp is more than 24 hours ago, the file is stale and the mirror client SHOULD warn the operator in log messages or other alerting, but MAY continue to process it otherwise.

4.5. Policy Restrictions

A mirror client MAY have a policy that restricts the processing of objects to certain object classes, or other limitations on which objects it processes.

If a mirror client has a policy that restricts object processing, this MUST be applied consistently to Snapshot Files and Delta Files from the moment the policy is enacted or modified.

5. Database Configuration File

REMOVE BEFORE PUBLICATION: this whole section is wide open to suggestions / ideas

5.1. Purpose

The Database Configuration File is provided by the IRR Database operator as configuration information for mirror clients. For each IRR Database run by an operator, it contains the current signing key, Snapshot Notification File URL and the IRR Database name. This JSON file MUST be placed on a well-known URI [RFC8615] on the "irr-nrtm.json" suffix on the authoritative domain name of the IRR Database operator and must be available over HTTPS.

The operator MUST update this file if any of these configuration details change.

5.2. Cache concerns

A mirror server may use caching infrastructure to cache the Database Configuration File and reduce the load of HTTPS requests. Unlike some of the other files, this file is both very small and quite rarely retrieved, limiting the need for and benefits of caching. Since this file is used by mirror clients to determine the current signing key, this SHOULD not cached for longer than one minute to support key rotations.

5.3. File format and validation

Example Database Configuration File:

[
  {
    "source": "EXAMPLE-A",
    "update_notification": "https://example.net/db/nrtmv4/example-a/notification.json",
    "key": "96..ae"
  },
  {
    "source": "EXAMPLE-B",
    "update_notification": "https://cdn.example.com/db/nrtmv4/example-b/notification.json",
    "key": "b6..3d",
  }
]

Note: the keys in this example are shortened because of formatting.

The following validation rules MUST be observed when creating or parsing Database Configuration Files:

6. Update Notification File

6.1. Purpose

The Update Notification File is generated by the mirror server and used by mirror clients to discover whether any changes exist between the state of the IRR mirror server and of the mirror client's. It also describes the location of the Snapshot File and incremental Delta Files. Finally, the generation timestamp can be used to detect whether the file is stale.

The mirror server MUST generate a new Update Notification File every time there are new deltas or snapshots and, even if there have been no changes, at least every 24 hours.

6.2. Cache concerns

A mirror server may use caching infrastructure to cache the Update Notification File and reduce the load of HTTPS requests.

However, since this file is used by mirror clients to determine whether any updates are available, the mirror server SHOULD ensure that this file is not cached for longer than one minute. An exception to this rule is that it is better to serve a stale Update Notification File rather than no Update Notification File.

6.3. File format and validation

Example Update Notification File:

{
  "nrtm_version": 4,
  "timestamp": "2022-01-00T15:00:00Z",
  "type": "notification",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 3,
  "snapshot": {
    "version": 2,
    "url": "https://example.com/ca128382-78d9-41d1-8927-1ecef15275be/nrtm-snapshot.2.047595d0fae972fbed0c51b4a41c7a349e0c47bb.json",
    "hash": "9a..86"
  }
  "deltas": [
    {
      "version": 1,
      "url": "https://example.com/ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.1.784a2a65aba22e001fd25a1b9e8544e058fbc703.json",
      "hash": "62..a2"
    }
    {
      "version": 2,
      "url": "https://example.com/ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.2.0f681f07cfab5611f3681bf030ec9f6fa3442fb0.json",
      "hash": "25..9a"
    }
    {
      "version": 3,
      "url": "https://example.com/ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.3.d9c194acbb2cb0d4088c9d8a25d5871cdd802c79.json",
      "hash": "b4..13"
    }
  ]
}

Note: hash values in this example are shortened because of formatting.

The following validation rules MUST be observed when creating or parsing Update Notification Files:

6.4. Signature

The contents of Update Notification File MUST be signed using Ed25519 [RFC8032]. The public key for this signature is published in the Database Configuration File. The signature of the Update Notification File MUST be published under the same path as the Update Notification File, appending ".sig".

7. Snapshot File

7.1. Purpose

The Snapshot File reflects the complete and current contents of all IRR objects in an IRR Database. Mirror clients MUST use this to initialize their local copy of the IRR Database.

7.2. Cache Concerns

A snapshot reflects the content of the IRR Database at a specific point in time; for that reason, it can be considered immutable data. Snapshot Files MUST be published at a URL that is unique to the specific session and version. The URL MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.

Because these files never change, they MAY be cached indefinitely. However, as snapshots are large and old snapshots will no longer be referred by newer Update Notification Files, it is RECOMMENDED that a limited interval is used in the order of hours or days.

To avoid race conditions where a mirror client retrieves an Update Notification File moments before it's updated, mirror servers SHOULD retain old Snapshot Files for at least 5 minutes after a new Update Notification File is published.

7.3. File format and validation

Example Snapshot File:

{
  "nrtm_version": 4,
  "type": "snapshot",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 2,
  "objects": [
    "route: 192.0.2.0/24\norigin: AS65530\nsource: EXAMPLE",
    "route: 2001:db8::/32\norigin: AS65530\nsource: EXAMPLE"
  ]
}

Note: IRR object texts in this example are shortened because of formatting.

The following validation rules MUST be observed when creating or parsing Snapshot Files:

8. Delta File

8.1. Purpose

A Delta File contains all changes for exactly one incremental update of the IRR Database. It may include new, modified and deleted objects. Delta Files can contain multiple alterations to multiple objects.

8.2. Cache Concerns

Deltas reflect the difference in content of the IRR Database from one version to another; for that reason, it can be considered immutable data. Delta Files MUST be published at a URL that is unique to the specific session and version. The URL MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.

To avoid race conditions where a mirror client retrieves an Update Notification File moments before it's updated, mirror servers SHOULD retain old Delta Files for at least 5 minutes after a new Update Notification File is published that no longer contains these Delta Files.

8.3. File format and validation

Example Delta File:

{
  "nrtm_version": 4,
  "type": "delta",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 3,
  "changes": [
    {
      "action": "delete",
      "existing_object": "route: 192.0.2.0/24\norigin: AS65530\nsource: EXAMPLE",
    }
    {
      "action": "add_modify",
      "new_object": "route: 2001:db8::/32\norigin: AS65530\nsource: EXAMPLE",
    }
  ]
}

Note: IRR object texts in this example are shortened because of formatting.

The following validation rules MUST be observed when creating or parsing Delta Files:

9. Operational Considerations

9.1. IRR object Validation

Throughout the years, various implementations of IRR servers have taken liberties with the various RFCs regarding RPSL. Implementations have introduced different new object classes, attributes and validation rules. Current IRR Databases also contain legacy objects which were created under different validation rules. In practice, there is no uniformly implemented standard for RPSL, but merely rough outlines partially documented in different places.

This has the potential to create interoperability issues. Some are addressed by NRTMv4, like having a consistent character set when mirroring data between implementations. However, some issues can not be addressed in this way, such as one implementation introducing a new object class that is entirely unknown to another implementation.

A mirror client SHOULD be able to handle unknown object classes and objects that are invalid according to its own validation rules, which may mean simply discarding them, without rejecting remaining objects or preventing future updates.

It is RECOMMENDED for mirror clients to log these cases, particularly those where an object was discarded due to violating validation rules. These cases create an inconsistency between the IRR objects of the server and client, and logs facilitate later analysis.

It is RECOMMENDED for mirror clients to be flexible where possible and reasonable when applying their own validation rules to IRR objects retrieved from mirror servers. For example, a route object with an origin attribute that is not a valid AS number can't be usefully interpreted. There is no way for an IRR server to correctly parse and index such an object. However, a route-set object whose name does not start with "RS-" [RFC2622], or an inetnum with an unknown extra "org" attribute, still allows the mirror client to interpret it unambiguously even if it does not meet the mirror client's own validation rules for authoritative records.

9.2. Intermediate mirror instances

An IRR Database generally has a single authoritative source. In some cases, an instance run by a third party will function as a kind of intermediate: both being a mirror client, mirroring IRR objects from the authoritative source, and simultaneously function as a mirror server to yet another mirror client.

There are various operational reasons for such a setup, such as the intermediate filtering certain records. Regardless of the reason, the mirror client and server function of an IRR server must be treated as separate processes. In particular, this means they MUST have separate session IDs. The intermediate server MUST NOT republish the same files it retrieved from the authoritative source with the same session ID.

9.3. Reading from local files

In the typical use case for NRTMv4, a mirror client retrieves files from an HTTPS endpoint. However, implementations MAY also support reading from files on the local filesystem instead, for when operators want to use a different method to retrieve or distribute the files. When reading from local files, mirror clients SHOULD still follow all validation rules, including the validation of the signature and hashes.

9.4. Public key rotation

It is RECOMMENDED that IRR Database operators rotate the signing key on their mirror server about once per year. To support this, the mirror server implementation MUST provide a way for operators to immediately generate a new Update Notification File on demand. To perform the rotation, the operator then:

Mirror clients will load the Update Notification File, find the signature to be invalid, retrieve the Database Configuration File and configure the new key. Note that key rotation will briefly but significantly increase the retrievals of the Database Configuration File, as almost every current mirror client will retrieve it within the minute after publication of the new Update Notification File.

10. Security Considerations

IRR objects serve many purposes, including automated network configuration and filtering. Manipulation of IRR objects can therefore have a significant security impact. However, security in existing protocols is mostly absent.

Before NRTMv4, the most common protocols for IRR Database mirroring are FTP for retrieving full snapshots, and NRTM version 3 for retrieving later changes. There are no provisions for integrity or authenticity, and there are various scenarios where mirroring may not be reliable.

NRTMv4 requires integrity verification. The Delta and Snapshot Files are verified using the SHA-256 hash in the Update Notification File, and the Update Notification File is verified using its signature file. Additionally, the channel security offered by HTTPS further limits security risks.

By allowing publication on any HTTPS endpoint, NRTMv4 allows for extensive scaling, and there are many existing techniques and services to protect against denial-of-service attacks. In contrast, NRTMv3 required mirror clients to directly query the IRR server instance with special whois queries. This scales poorly, and there are no standard protections against denial-of-service available.

The HTTPS endpoint used for NRTMv4 MUST be configured according to the best practices in [RFC7525].

11. IANA Considerations

This document requests IANA to ...

12. Acknowledgments

The authors would like to thank ... and ... for their helpful review of this document.

13. References

13.1. Normative References

[RFC2045]
Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, DOI 10.17487/RFC2045, , <https://www.rfc-editor.org/info/rfc2045>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC2622]
Alaettinoglu, C., Villamizar, C., Gerich, E., Kessens, D., Meyer, D., Bates, T., Karrenberg, D., and M. Terpstra, "Routing Policy Specification Language (RPSL)", RFC 2622, DOI 10.17487/RFC2622, , <https://www.rfc-editor.org/info/rfc2622>.
[RFC3339]
Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, DOI 10.17487/RFC3339, , <https://www.rfc-editor.org/info/rfc3339>.
[RFC4122]
Leach, P., Mealling, M., and R. Salz, "A Universally Unique IDentifier (UUID) URN Namespace", RFC 4122, DOI 10.17487/RFC4122, , <https://www.rfc-editor.org/info/rfc4122>.
[RFC4648]
Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, , <https://www.rfc-editor.org/info/rfc4648>.
[RFC7525]
Sheffer, Y., Holz, R., and P. Saint-Andre, "Recommendations for Secure Use of Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, , <https://www.rfc-editor.org/info/rfc7525>.
[RFC8032]
Josefsson, S. and I. Liusvaara, "Edwards-Curve Digital Signature Algorithm (EdDSA)", RFC 8032, DOI 10.17487/RFC8032, , <https://www.rfc-editor.org/info/rfc8032>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[RFC8615]
Nottingham, M., "Well-Known Uniform Resource Identifiers (URIs)", RFC 8615, DOI 10.17487/RFC8615, , <https://www.rfc-editor.org/info/rfc8615>.
[SHS]
National Institute of Standards and Technology, "Secure Hash Standard", , <http://csrc.nist.gov/publications/fips/fips180-4/fips-180-4.pdf>.

13.2. Informative References

[RFC8182]
Bruijnzeels, T., Muravskiy, O., Weber, B., and R. Austein, "The RPKI Repository Delta Protocol (RRDP)", RFC 8182, DOI 10.17487/RFC8182, , <https://www.rfc-editor.org/info/rfc8182>.

Authors' Addresses

Sasha Romijn
Reliably Coded
Amsterdam
Netherlands
Job Snijders
Fastly
Amsterdam
Netherlands
Edward Shryane
RIPE NCC
Amsterdam
Netherlands
Stavros Konstantaras
AMS-IX
Amsterdam
Netherlands