Special Loop Address

RTT - Round Trip Time TTL - Time to Live BFD - BiDirectional Failure Detection LFM - Link Fault Management ICMP - Internet Control Message Protocol

Real time monitoring of WAN or MAN link quality presents a real operational challenge. The common use of circuit emulation techniques by carriers makes detection of the circuits degradation difficult. Very often such reduced link quality results in increased queuing times or packet drops beyond SLA guarantees. Furthermore, the characteristics of link degradation is different from link to link. The problem space described above is further complicated due to the following reasons: Link anomalies may not occur at the same uniform rate or be of the same constant and continuous pattern. This transient characteristic maybe a function of load or other temporary problems for example transport network over-subscription. Encountered degradated service behavior may not translate to link errors or packet discards on either end of the suspected link because the emulated link consisting of multiple independent L2 segments in the carrier's network. Currently available tools on the circuit endpoints (usually routers) do not allow easy way to diagnose circuit health. Tools used today to detect link issues include: Creating hardware or software loops manually - this results in the actual link under test to be taken out of service. Test traffic is then sent through the link and based on the results of the test, link quality issues are detected. Regular pings/probes on directly connected links between routers/network devices - Depending on the size of the probe packets and the rate at which they are sent between the network devices and the loss, the link issues are detected. The issue with this approach is that network processor on the router has to process all these packets. This causes an additional processing load on the routers. BFD, IP protocol hellos etc are based on detecting neighbor state based on tiny and lightweight hellos. Such probes were designed for fast detection of end-to-end link state events .. not to evaluate link quality. If say N hellos send in T interval are lost it is an indication about link or peer down event. The layer 2 OAM tools are not capable of addressing the requirements since by definition an emulated link consists of number of different L2 links hidden by the emulation layer and its encapsulation. L2 OAM could only indicate potential problems within single layer 2 link. They are light weight and some of these issues can only be detected at various levels of data rates (within agreed SLAs) transiting via such links.

The following are some of the key considerations required to be addressed in an alternative diagnostics solutions: The testing should be atomic in nature - the UUT in this document is a single p2p link. The test should not be subject to any alterations by externally injected packets The probe packets should never be able to transit L3 node to any other L3 node The level of diagnostics data should be configurable such that operator is able to inject anywhere from 0.1% to 100% of test load of a given max link capacity with build in automatic consideration of existing average of production traffic load (unless link is considered as taken out-of-service). The duration of the test traffic should be either configurable by the operator or controlled by built-in detection heuristics. The frequency of the test traffic should be either configurable by the operator or controlled by built-in detection heuristics. Probes should not be subject to process switching by the route processors on either end of the link during the burst. The solution should strive to minimize amount of required protocol extensions for as easy as possible inter-operability characteristics. In the topologies where Link Aggregation is used, the aggregated bandwidth of the link should be considered instead of the individual links. The probe accounting should be recorded as total of all link members. Probe's hashing should follow normal data plane load balancing rules as configured on the directly connected peering routers.

The mechanism for the set of proposed requirements can be constructed by combining two standards based protocol elements: TTL field processing and special purpose IPv4 or IPv6 loop addresses. Special purpose loop address will allow to setup a scoped link based loop and TTL field can be used to limit the loop duration. The special purpose loop address for this purpose can be subset of the link local range - 169.254/16 for IPv4 [RFC 3927] or FE80::/64 for IPv6 [RFC 4291] or it could be taken from an alternative pool if IETF process suggests so. Selected and allocated special purpose loop address would be therefor kept and maintained by IANA IPv4 or IPv6 Special-Purpose Address Registries. Routers must not forward any packets with loop source or destination addresses to links other then the link packet arrived on. The IPv4/IPv6 loop address MAY BE associated with numbered IP addresses for the given link or with link local addresses. The resolution to MAC address of L2 rewrites would be resolved locally through corresponding L3 adjacency addresses. IPv4/IPv6 test packet is directed towards L3 neighbor with even TTL value.

The following is considered as a high level description of proposed solution: Two routers R1 and R2 connected together by link L1 Average RTT between R1 and R2 on link L1 is 5ms R1 and R2 have IP connectivity with each other on 10.10.10.0/30 numbered link. R1 has been configured with IP an address of 10.10.10.1 and R2 has been configured with an IP address of 10.10.10.2 For the purpose of a test an IP loop address is configured on R1 and R2 to create local link loops. For the purpose of this illustration the loop address has been named as L.O.O.P/32 The following IPv4 packet has been injected from R1: Source IP address: 10.10.10.1 Destination IP address: L.O.O.P TTL = 254 payload optional ... (to be discussed by WG) Test sequence: Packet arrives at R2 and TTL is decremented following by destination IP lookup and re-injection towards R1 Packet keeps looping till the TTL expires on R1. Upon TTL expiration an ICMP TTL EXPIRED error message is being sent to the source of the original packet (10.10.10.1). The ICMP message contains the header information of the original packet Observations: A test probe packet has been amplified 254 times for a short time An ICMP TTL expired message is indicative that result of the test can be described as: probe packets were not dropped No ICMP TTL message implies that one copy of the original packet was lost while it was looping between two routers. No reception of ICMP TTL indicates potential issues with the link provided that test sequence was assured never exceed agreed SLAs for a given link. Ability to send multiple packets of different sizes on the link with inherently controlled TTL loop can results in expected burst of control/probe traffic on the link under test Such probe burst can be programmed to get to a certain % of the link speed for a short time Based on fine tuned testing scenario allowing to fill the bandwidth up to a certain % of link capacity the count of packets originally sent by router R1 should be the same as the number of ICMP TTL expired messages. If the count of packets originally sent by router is the same as the number of ICMP TTL expired messages then the test is successful. If however the number of ICMP TTL expired messages is less than the count of packets originally sent by the router then the test is unsuccessful proving potential problems with the link. A test probe packet with even initial TTL value will generate a TTL time expired ICMP message on the originating router. A test probe packet with odd initial TTL value will generate a TTL time expired ICMP message on the neighboring router. It is RECOMMENDED that the test probe is sent with even initial TTL value. So, ICMP messages are not traversing the link under test. It is RECOMMENDED that a special payload structure is used for these test probes with sequence numbers. When the TTL expires and an ICMP message is generated, the IP header + 64 bits from original packet gets copied to ICMP message [RFC792]. This can be used for associating the ICMP message and the test. The MTU of the test probes can be adjusted up to maximum MTU value of the link. Fragmentation of probe packets SHOULD be avoided.

Analysis of the proposed solution against the actual new test methodology requirements: Provides means to potentially fill up the part of link bandwidth very rapidly due to inherent amplification especially with high initial TTL value. The fill level of the test traffic is a function of: Initial packet size (higher the packet size the higher the fill level), Initial TTL value (higher the TTL value, higher the multiplicative factor for packets and hence higher the fill level), Initial number of packets sent (the more the packets sent the more the fill level) and MTU of the probe packets. Test can be run together with production traffic. There is no impact on production traffic neither there is any requirement to stop production traffic in order to perform the test. The amplification of the packets and looping happens as a part of inherent forwarding in the routers. This solution does not require a special process in software or hardware to send the test probes between the two routers as special purpose loop address would be part of standard FIB tables. This mechanism is light-weight and does not require any new software implementations. Potential for local vendor's optimizations however is still there in the area of segmenting TTL equal zero errors of probes from other transit uses of TTL equal zero errors or in the space of result presentation to the operators.

Initial packet size and rate are important to determine the test fill level for the link. The test packet loops the same number of times as the original TTL value of the original packet. The time it takes for the original packet to come back to the original router is the RTT (Round Trip Time) value between two routers. Under the assumptions that: RTT of link under test is 1ms, link speed 1 Gb/s, packet size of test packet is 1536 bytes, TTL on original packet is set to 254, would result in the test packets looping for 254 ms. Under the above assumptions it is easy to calculate that in order fill 1 Gb/s link to 100% 81 such probe packets need to be injected. Likewise in order to fill such link to 20% of its capacity 16 probe packets are required.

Since injected test packets are regular IP packets they can be marked with any class of service. As a result the test probes similar to actual data will be processed based on the real QoS configuration and will be subject to treatment defined for a given packet class. That allows both prioritization as well as de-prioritization of a given set of test probes.

The payload of the test packets can be of any IP protocol. The link fill levels is also a function of Inter-packet gap of the test and the RTT of that link. Deterministic fill levels can only be derived by accounting for RTT of the link under test.

It is expected that link testing methodology described in this document will be accessible by I2RS channel as well as extensions to YANG models will be defined for both setting and retrieval of the data.

This document requires IANA to allocate and maintain following Special Purpose IP Addresses: IPv4 Special Purpose Loop Address and maintain it the IANA IPv4 Special-Purpose Address Registry [RFC5735] IPv6 Special Purpose Loop Address and maintain it the IANA IPv6 Special-Purpose Address Registry [RFC5156]

While the proposed mechanism does not define any new protocols nor protocol extensions of already existing specifications it does relay on the TTL-expiry notifications. Such notifications must be enabled and must not be limited in any way for the specific class of probe packets. It is highly recommended that test destinations LOOP addresses are not routeable beyond their locally attached links. Using IPv4/IPv6 special purpose loop addresses will address that.

Authors would like to thank Truman Boyes and Leo Pang for their valuable input.