IPv6 Operations E. Vyncke Internet-Draft Cisco Intended status: Informational R. Léas Expires: 5 September 2022 J. Iurman Université de Liège 4 March 2022 Just Another Measurement of Extension header Survivability (JAMES) draft-vyncke-v6ops-james-00 Abstract In 2016, [RFC7872] has measured the drop of packets with IPv6 extension headers. This document presents a slightly different methodology with more recent results. It is still work in progress. About This Document This note is to be removed before publishing as an RFC. The latest revision of this draft can be found at https://evyncke.github.io/v6ops-james/draft-vyncke-v6ops-james.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-vyncke-v6ops-james/. Discussion of this document takes place on the IPv6 Operations Working Group mailing list (mailto:v6ops@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/v6ops/. Source for this draft and an issue tracker can be found at https://github.com/evyncke/v6ops-james. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Vyncke, et al. Expires 5 September 2022 [Page 1] Internet-Draft JAMES March 2022 This Internet-Draft will expire on 5 September 2022. Copyright Notice Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Measurements . . . . . . . . . . . . . . . . . . . . . . . . 3 3.1. Vantage Points . . . . . . . . . . . . . . . . . . . . . 3 3.1.1. Tested Autonomous Systems . . . . . . . . . . . . . . 4 3.1.2. Tested Extension Headers . . . . . . . . . . . . . . 6 3.1.3. Drop attribution to AS . . . . . . . . . . . . . . . 7 3.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2.1. Routing Header . . . . . . . . . . . . . . . . . . . 8 3.2.2. Hop-by-Hop Options Headers . . . . . . . . . . . . . 9 3.2.3. Destination Options Headers . . . . . . . . . . . . . 10 3.2.4. Fragmentation Header . . . . . . . . . . . . . . . . 11 3.2.5. No drop at all . . . . . . . . . . . . . . . . . . . 11 3.3. Summary of the collaborating parties measurements . . . . 12 4. Security Considerations . . . . . . . . . . . . . . . . . . . 12 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.1. Normative References . . . . . . . . . . . . . . . . . . 13 6.2. Informative References . . . . . . . . . . . . . . . . . 13 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 1. Introduction In 2016, [RFC7872] has measured the drop of packets with IPv6 extension headers on their transit over the global Internet. This document presents a slightly different methodology with more recent results. Since then, [I-D.draft-ietf-opsec-ipv6-eh-filtering] has provided some recommendations for filtering transit traffic, there may be some changes in providers policies. Vyncke, et al. Expires 5 September 2022 [Page 2] Internet-Draft JAMES March 2022 It is still work in progress, but the authors wanted to present some results at IETF-113 (March 2022). 2. Methodology In a first phase, the measurement is done between collaborating IPv6 nodes, a.k.a. vantage points, spread over the Internet and multiple Autonomous Systems (ASs). As seen in Section 3.1.1, the source/destination/transit ASs include some "tier-1" providers per [TIER1], so, they are probably representative of the global Internet core. Relying on collaborating nodes has some benefits: * propagation can be measured even in the absence of any ICMP message or reply generated by the destination; * traffic timing can be measured accurately to answer whether extension headers are slower than plain IP6 packets; * traffic can be captured into .pcap [I-D.draft-ietf-opsawg-pcap] file at the source and at the destination for later analysis. Future phases will send probes to non-collaborating nodes with a much reduced probing speed. The destination will include [ALEXA] top-n websites, popular CDN, as well as random prefix from the IPv6 global routing table. A revision of this IETF draft will describe those experiments. 3. Measurements 3.1. Vantage Points Several servers were used worldwide (albeit missing Africa and China as authors were unable to find IPv6 servers in these regions). Table 1 lists all the vantage points together with their AS number and country. Vyncke, et al. Expires 5 September 2022 [Page 3] Internet-Draft JAMES March 2022 +========+===================+==============+===================+ | ASN | AS Name | Country code | Location | +========+===================+==============+===================+ | 7195 | Edge Uno | AG | Buenos Aires | +--------+-------------------+--------------+-------------------+ | 12414 | NL-SOLCON SOLCON | NL | Amsterdam | +--------+-------------------+--------------+-------------------+ | 14061 | Digital Ocean | CA | Toronto, ON | +--------+-------------------+--------------+-------------------+ | 14061 | Digital Ocean | USA | New York City, NY | +--------+-------------------+--------------+-------------------+ | 14601 | Digital Ocean | DE | Francfort | +--------+-------------------+--------------+-------------------+ | 14601 | Digital Ocean | IN | Bangalore | +--------+-------------------+--------------+-------------------+ | 14601 | Digital Ocean | SG | Singapore | +--------+-------------------+--------------+-------------------+ | 16276 | OVH | AU | Sydney | +--------+-------------------+--------------+-------------------+ | 16276 | OVH | PL | Warsaw | +--------+-------------------+--------------+-------------------+ | 44684 | Mythic Beasts | UK | Cambridge | +--------+-------------------+--------------+-------------------+ | 47853 | Hostinger | US | Ashville, NC | +--------+-------------------+--------------+-------------------+ | 60011 | MYTHIC-BEASTS-USA | US | Fremont, CA | +--------+-------------------+--------------+-------------------+ | 198644 | GO6 | SI | Ljubljana | +--------+-------------------+--------------+-------------------+ Table 1: All vantage AS 3.1.1. Tested Autonomous Systems During first phase (traffic among fully meshed collaborative nodes), our probes have collected data on the following ASs: +===========+======================================+==========+ | AS Number | AS Description | Comment | +===========+======================================+==========+ | 174 | COGENT-174, US | Tier-1 | +-----------+--------------------------------------+----------+ | 1299 | TWELVE99 Twelve99, Telia Carrier, SE | Tier-1 | +-----------+--------------------------------------+----------+ | 2914 | NTT-COMMUNICATIONS-2914, US | Tier-1 | +-----------+--------------------------------------+----------+ | 3320 | DTAG Internet service provider | Tier-1 | | | operations, DE | | Vyncke, et al. Expires 5 September 2022 [Page 4] Internet-Draft JAMES March 2022 +-----------+--------------------------------------+----------+ | 3356 | LEVEL3, US | Tier-1 | +-----------+--------------------------------------+----------+ | 4637 | ASN-TELSTRA-GLOBAL Telstra Global, | Regional | | | HK | Tier | +-----------+--------------------------------------+----------+ | 4755 | TATACOMM-AS TATA Communications | | | | formerly VSNL is Leading ISP, IN | | +-----------+--------------------------------------+----------+ | 5603 | SIOL-NET Telekom Slovenije d.d., SI | | +-----------+--------------------------------------+----------+ | 6453 | Tata Communication | Tier-1 | +-----------+--------------------------------------+----------+ | 6762 | SEABONE-NET TELECOM ITALIA SPARKLE | Tier-1 | | | S.p.A., IT | | +-----------+--------------------------------------+----------+ | 6939 | HURRICANE, US | Regional | | | | Tier | +-----------+--------------------------------------+----------+ | 7195 | EDGEUNO SAS, CO | | +-----------+--------------------------------------+----------+ | 8447 | A1TELEKOM-AT A1 Telekom Austria AG, | | | | AT | | +-----------+--------------------------------------+----------+ | 9498 | BBIL-AP BHARTI Airtel Ltd., IN | | +-----------+--------------------------------------+----------+ | 12414 | NL-SOLCON SOLCON, NL | | +-----------+--------------------------------------+----------+ | 14061 | DIGITALOCEAN-ASN, US | | +-----------+--------------------------------------+----------+ | 16276 | OVH, FR | | +-----------+--------------------------------------+----------+ | 21283 | A1SI-AS A1 Slovenija, SI | | +-----------+--------------------------------------+----------+ | 34779 | T-2-AS AS set propagated by T-2 | | | | d.o.o., SI | | +-----------+--------------------------------------+----------+ | 44684 | MYTHIC Mythic Beasts Ltd, GB | | +-----------+--------------------------------------+----------+ | 60011 | MYTHIC-BEASTS-USA, GB | | +-----------+--------------------------------------+----------+ | 198644 | GO6, SI | | +-----------+--------------------------------------+----------+ Table 2: All AS (source/destination/transit) Vyncke, et al. Expires 5 September 2022 [Page 5] Internet-Draft JAMES March 2022 The table attributes some tier qualification to some ASs based on the Wikipedia page [TIER1], but there is no common way to decide who is a tier-1. Based on some CAIDA research, all the above (except GO6, which is a stub network) are transit providers. While this document lists some operators, the intent is not to build a wall of fame or a wall of shame but more to get an idea about which kind of providers drop packets with extension headers and how widespread the drop policy is enforced and where, i.e., in the access provider or in the core of the Internet. 3.1.2. Tested Extension Headers In the first phase among collaborating vantage points, packets always contained either a UDP payload or a TCP payload, the latter is sent with only the SYN flag set and with data as permitted by section 3.4 of [RFC793] (2nd paragraph). A usual traceroute is done with only the UDP/TCP payload without any extension header with varying hop- limit in order to learn the traversed routers and ASs. Then, several UDP/TCP probes are sent with a set of extension headers: * hop-by-hop and destination options header containing: - one PadN option for an extension header length of 8 octets, - one unknown option with the "discard" bits for an extension header length of 8 octets, - several PaDN options for an extension header length of 256 octets, - one unknown option (two sets with "discard" and "skip" bits) for an extension header length of 256 octets, - one unknown option (two sets with "discard" and "skip" bits) for an extension header length of 512 octets. * routing header with routing types from 0 to 6 inclusive (except for type 3); * atomic fragment header (i.e., M-flag = 0 and offset = 0) of varying frame length 512, 1280, and 1500 octets; * non-atomic first fragment header (i.e., M-flag = 1 and offset = 0) of varying frame length 512, 1280, and 1500 octets; * authentication header with dummy SPI followed by UDP/TCP header and a 38 octets payload. Vyncke, et al. Expires 5 September 2022 [Page 6] Internet-Draft JAMES March 2022 In the above, length is the length of the extension header itself except for the fragmentation header where the length is the IP packet length (i.e., including the IPv6, and TCP/UDP headers + payload). For hop-by-hop and destination options headers, when required multiple PadN options were used in order to bypass some Linux kernels that consider a PadN larger than 8 bytes is an attack, see section 5.3 of [BCP220], even if multiple PadN options violates section 2.1.9.5 of [RFC4942]. Next phases will also include packets without UDP/TCP transport header but with Next-Header being: * 59, "no next header" see section 4.7 of [RFC8200]; * 143, "ethernet" see section 4.9 of [RFC8986]. 3.1.3. Drop attribution to AS Comparing the traceroutes with and without extension headers allows the attribution of a packet drop to one AS. But, this is not an easy task as inter-AS links often use IPv6 address of only one AS (if not using link-local per [RFC7704]). This document uses the following algorithm to attribute the drop to one AS for packet sourced in one AS and then having a path traversing AS#foo just before AS#bar: * if the packet drop happens at the first router (i.e., hop limit == 1 does not trigger an ICMP hop-limit exceeded), then the drop is assumed to this AS as it is probably an ingress filter on the first router (i.e., the hosting provider in most of the cases - except if collocated with an IXP). * if the packet drop happens in AS#foo after one or more hop(s) in AS#bar, then the drop is assumed to be in AS#foo ingress filter on a router with an interface address in AS#foo * if the packet drop happens in AS#bar after one or more hop(s) in AS#bar before going to AS#foo, then the drop is assumed to be in AS#foo ingress filter on a router with an interface address in AS#bar Vyncke, et al. Expires 5 September 2022 [Page 7] Internet-Draft JAMES March 2022 In several cases, the above algorithm was not possible (e.g., some intermediate routers do not generate an ICMP unreachable hop limit exceeded even in the absence of any extension headers), then the drop is not attributed. Please also note that the goal of this document is not to 'point fingers to operators' but more to evaluate the potential impact. I.e., a tier-1 provider dropping packets with extension headers has a much bigger impact on the Internet traffic than an access provider. Future revision of this document will use the work of [MLAT_PEERING]. 3.2. Results This section presents the current results out of phase 1 (collaborating vantage points) testing. About 4860 experiments were run, one experiment being defined by sending packets between 2 vantage points with hop-limit varying from 1 to the number of hops between the two vantage points and for all the extension headers described in Section 3.1.2. 3.2.1. Routing Header The following table lists all routing header types and the percentage of experiments that were successful, i.e., packets with routing header reaching their destination: +=====================+===========================================+ | Routing Header Type | %-age of experiments reaching destination | +=====================+===========================================+ | 0 | 80.9% | +---------------------+-------------------------------------------+ | 1 | 99.5% | +---------------------+-------------------------------------------+ | 2 | 99.5% | +---------------------+-------------------------------------------+ | 3 | not tested | +---------------------+-------------------------------------------+ | 4 | 69.0% | +---------------------+-------------------------------------------+ | 5 | 99.5% | +---------------------+-------------------------------------------+ | 6 | 99.3% | +---------------------+-------------------------------------------+ Table 3 Vyncke, et al. Expires 5 September 2022 [Page 8] Internet-Draft JAMES March 2022 The table below lists the few ASs that drop packets with the routing header type 0 (the original source routing header, which is now deprecated). +===========+================+ | AS Number | AS description | +===========+================+ | 6939 | HURRICANE, US | +-----------+----------------+ Table 4 It is possibly due to a strict implementation of [RFC5095] but it is expected that no packet with routing header type 0 would be transmitted anymore. So, this is not surprising. The table below lists the few ASs that drop packets with the routing header type 4 (Segment Routing Header [RFC8754]). +===========+================+ | AS Number | AS description | +===========+================+ | 16276 | OVH, FR | +-----------+----------------+ Table 5 This drop of SRH was to be expected as SRv6 is specified to run only in a limited domain. Other routing header types (1 == deprecated NIMROD [RFC1753], 2 == mobile IPv6 [RFC6275], and even 5 == CRH-16 and 6 == CRH- 32[I-D.draft-bonica-6man-comp-rtg-hdr]) can be transmitted over the global Internet without being dropped (assuming that the 0.5% of dropped packets are within the measurement error). 3.2.2. Hop-by-Hop Options Headers Many ASs drop packets containing either hop-by-hop options headers per table below: Vyncke, et al. Expires 5 September 2022 [Page 9] Internet-Draft JAMES March 2022 +===============+========+======================+ | Option Type | Length | %-age of experiments | | | | reaching destination | +===============+========+======================+ | Skip | 8 | 5.8% | +---------------+--------+----------------------+ | Discard | 8 | 0.0% | +---------------+--------+----------------------+ | Skip one | 256 | 1.9% | | large PadN | | | +---------------+--------+----------------------+ | Skip multiple | 256 | 0.0% | | PadN | | | +---------------+--------+----------------------+ | Discard | 256 | 0.0% | +---------------+--------+----------------------+ | Skip | 512 | 1.9% | +---------------+--------+----------------------+ | Discard | 512 | 0.0% | +---------------+--------+----------------------+ Table 6 It appears that hop-by-hop options headers cannot reliably traverse the global Internet; only small headers with 'skippable' options have some chances. If the hop-by-hop option has the 'discard' bits, it is dropped per specification. 3.2.3. Destination Options Headers Many ASs drop packets containing destination options headers per table below: +========+===============+======================+ | Length | Multiple PadN | %-age of experiments | | | | reaching destination | +========+===============+======================+ | 8 | No | 99.2% | +--------+---------------+----------------------+ | 256 | No | 2.6% | +--------+---------------+----------------------+ | 256 | Yes | 2.6% | +--------+---------------+----------------------+ | 512 | No | 2.6% | +--------+---------------+----------------------+ Table 7 Vyncke, et al. Expires 5 September 2022 [Page 10] Internet-Draft JAMES March 2022 The measurement did not find any impact of the discard/skip bits in the destination headers options, probably because the routers do not look inside the extension headers into the options. The use of a single large PadN or multiple 8-octet PadN options does not influence the result. 3.2.4. Fragmentation Header The propagation of two kinds of fragmentation headers was analysed: atomic fragment (offset == 0 and M-flag == 0) and plain first fragment (offset == 0 and M-flag == 1). The table below displays the propagation differences. +============+===========================================+ | M-flag | %-age of experiments reaching destination | +============+===========================================+ | 0 (atomic) | 70.2% | +------------+-------------------------------------------+ | 1 | 99.0% | +------------+-------------------------------------------+ Table 8 The size of the overall IP packets (512, 1280, and 1500 octets) does not have any impact on the propagation. 3.2.5. No drop at all Finally, some ASs do not drop transit traffic (except for routing header type 0) and follow the recommendations of [I-D.draft-ietf-opsec-ipv6-eh-filtering]. This list includes tier-1 transit providers (using the "regional" tag per [TIER1]): Vyncke, et al. Expires 5 September 2022 [Page 11] Internet-Draft JAMES March 2022 +===========+=======================================+===============+ | AS Number | AS Description | Comment | +===========+=======================================+===============+ | 4637 | ASN-TELSTRA-GLOBAL Telstra Global, HK | Regional | | | | Tier | +-----------+---------------------------------------+---------------+ | 4755 | TATACOMM-AS TATA Communications | | | | formerly VSNL is Leading ISP, IN | | +-----------+---------------------------------------+---------------+ | 6762 | SEABONE-NET TELECOM ITALIA SPARKLE | Tier-1 | | | S.p.A., IT | | +-----------+---------------------------------------+---------------+ | 14061 | DIGITALOCEAN-ASN, US | | +-----------+---------------------------------------+---------------+ | 21283 | A1SI-AS A1 Slovenija, SI | | +-----------+---------------------------------------+---------------+ | 60011 | MYTHIC-BEASTS-USA, GB | | +-----------+---------------------------------------+---------------+ Table 9 3.3. Summary of the collaborating parties measurements While the analysis has areas of improvement (geographical distribution and impact on latency), it appears that: * authentication and non-atomic fragmentation headers can traverse the Internet; * only routing headers types 0 and 4 experiment problems over the Internet, other types have no problems; * hop-by-hop options headers do not traverse the Internet; * destination options headers are not reliable enough especially large size ones. Of course, the next phase of measurement with non-collaborating parties will probably give another view. 4. Security Considerations While active probing of the Internet may be considered as an attack, this measurement was done among collaborating parties and using the probe attribution technique described in [I-D.draft-vyncke-opsec-probe-attribution]. Vyncke, et al. Expires 5 September 2022 [Page 12] Internet-Draft JAMES March 2022 5. IANA Considerations This document has no IANA actions. 6. References 6.1. Normative References [RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, DOI 10.17487/RFC0793, September 1981, . [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017, . [RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, February 2021, . 6.2. Informative References [ALEXA] "The top 500 sites on the web", n.d., . [BCP220] Chown, T., Loughney, J., and T. Winters, "IPv6 Node Requirements", BCP 220, RFC 8504, January 2019. [I-D.draft-bonica-6man-comp-rtg-hdr] Bonica, R., Kamite, Y., Alston, A., Henriques, D., and L. Jalil, "The IPv6 Compact Routing Header (CRH)", Work in Progress, Internet-Draft, draft-bonica-6man-comp-rtg-hdr- 27, 15 November 2021, . [I-D.draft-ietf-opsawg-pcap] Harris, G. and M. C. Richardson, "PCAP Capture File Format", Work in Progress, Internet-Draft, draft-ietf- opsawg-pcap-00, 25 October 2021, . Vyncke, et al. Expires 5 September 2022 [Page 13] Internet-Draft JAMES March 2022 [I-D.draft-ietf-opsec-ipv6-eh-filtering] Gont, F. and W. (. Liu, "Recommendations on the Filtering of IPv6 Packets Containing IPv6 Extension Headers at Transit Routers", Work in Progress, Internet-Draft, draft- ietf-opsec-ipv6-eh-filtering-08, 3 June 2021, . [I-D.draft-vyncke-opsec-probe-attribution] Vyncke, É., Donnet, B., and J. Iurman, "Attribution of Internet Probes", Work in Progress, Internet-Draft, draft- vyncke-opsec-probe-attribution-01, 3 March 2022, . [MLAT_PEERING] Giotsas, V., Zhou, S., Luckie, M., and K. Claffy, "Inferring Multilateral Peering", DOI 10.1145/2535372.2535390, December 2013, . [RFC1753] Chiappa, N., "IPng Technical Requirements Of the Nimrod Routing and Addressing Architecture", RFC 1753, DOI 10.17487/RFC1753, December 1994, . [RFC4942] Davies, E., Krishnan, S., and P. Savola, "IPv6 Transition/ Co-existence Security Considerations", RFC 4942, DOI 10.17487/RFC4942, September 2007, . [RFC5095] Abley, J., Savola, P., and G. Neville-Neil, "Deprecation of Type 0 Routing Headers in IPv6", RFC 5095, DOI 10.17487/RFC5095, December 2007, . [RFC6275] Perkins, C., Ed., Johnson, D., and J. Arkko, "Mobility Support in IPv6", RFC 6275, DOI 10.17487/RFC6275, July 2011, . [RFC7704] Crocker, D. and N. Clark, "An IETF with Much Diversity and Professional Conduct", RFC 7704, DOI 10.17487/RFC7704, November 2015, . Vyncke, et al. Expires 5 September 2022 [Page 14] Internet-Draft JAMES March 2022 [RFC7872] Gont, F., Linkova, J., Chown, T., and W. Liu, "Observations on the Dropping of Packets with IPv6 Extension Headers in the Real World", RFC 7872, DOI 10.17487/RFC7872, June 2016, . [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, . [TIER1] "Tier 1 network", n.d., . Acknowledgments The authors want to thank Sander Steffann and Jan Zorz for allowing the free use of their labs. Other thanks to Fernando Gont who indicated a nice IPv6 hosting provider in South America. Special thanks as well to Professor Benoit Donnet for his support and advices. This document would not have existed without his support. Authors' Addresses Éric Vyncke Cisco De Kleetlaan 64 1831 Diegem Belgium Email: evyncke@cisco.com Raphaël Léas Université de Liège Liège Belgium Email: raphael.leas@student.uliege.be Justin Iurman Université de Liège Belgium Email: justin.iurman@uliege.be Vyncke, et al. Expires 5 September 2022 [Page 15]