Fraunhofer FKIE

henning.rogge@fkie.fraunhofer.de http://www.fkie.fraunhofer.de

INRIA

Emmanuel.Baccelli@inria.fr http://www.emmanuelbaccelli.org/

Routing Area MANET metric ad hoc network MANET routing IP networks OLSR ETT ETX Funkfeuer This document specifies an directional airtime link metric for usage in OLSRv2.

One of the major shortcomings of OLSR is the missing of a link cost metric between mesh nodes. Operational experience with mesh networks gathered since the standardization of OLSR has revealed that wireless networks links can have highly variable and heterogeneous properties. This makes a hopcount metric insufficient for effective mesh routing. Based on this experience, OLSRv2 integrates the concept of link metrics directly into the core specification of the routing protocol. The OLSRv2 routing metric is an external process, it can be any kind of dimensionless additive cost function which reports to the OLSRv2 protocol. Since 2004 the OLSR.org implementation of OLSR included an Estimated Transmission Count (ETX) metric as a proprietary extension. While this metric is not perfect, it proved to be sufficient for a long time for Community Mesh Networks (). But the increasing maximum data rate of IEEE 802.11 made the ETX metric less efficient than in the past, which is one reason to move to a different metric. This document describes a Directional Airtime routing metric for OLSRv2, a successor of the OLSR.org routing metric for . It takes both the loss rate and the link speed into account to provide a more accurate picture of the mesh network links.

The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT','SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in . The terminology introduced in , and , including the terms "packet", "message" and "TLV" are to be interpreted as described therein. Additionally, this document uses the following terminology and notational conventions: - a first in, first out queue of integers. - the most recent element in the queue. - adds a new element to the TAIL of the queue. - removes the HEAD element of the queue - an operation which returns the sum of all elements in a QUEUE. - an operation which returns the positive distance between two elements of the circular sequence number space defined in section 5.1 of . Its value is either (new - old) if this result is positive, or else its value is (new - old + 65536). - the maximum of a and b. - a value not in the normal value range of a variable. Might be -1 for this protocol. - the time a transmitted packet blocks the link layer, e.g., a wireless link. - Expected Transmission Count, a link metric proportional to the number of transmissions to successfully send an IP packet over a link. - Estimated Travel Time, a link metric proportional to the amount of airtime needed to transmit an IP packet over a link, not considering layer-2 overhead created by preamble, backoff time and queuing. - Directional Airtime Metric, the link metric described in this document, which is a directional variant of ETT. It does not take reverse path loss into account.

The Directional Airtime Metric was designed and tested in wireless IEEE 802.11 mesh networks. These networks employ link layer retransmission to increase the delivery probability and multiple unicast data rates. The metric must learn about the unicast data rate towards each one-hop neighbor from an external process, either by configuration or by an external measurement process. This measurement could be done by gathering cross-layer data from the operating system or an external daemon like DLEP , but also by indirect layer-3 measurements like packet-pair. If control traffic is used to determine the link packet loss, the administrator should take care that link layer multicast transmission do not not have a higher reception probability than the slowest unicast transmission. It might be necessary to increase the data-rate of the multicast transmissions, e.g. set the multicast data-rate to 6 MBit/s if you use IEEE 802.11g only. The metric can only handle a certain range of packet loss and unicast data-rate. Maximum packet loss is "ETX 4" (1 of 4 packets is successfully sent to the receiver, without link layer retransmissions), the unicast data-rate can be between 1024 Bit/s and 4 GBit/s. The metric has been designed for data-rates of 1 MBit/s and hundreds of MBit/s.

The Directional Airtime Metric has been inspired by the publications on the ETX and ETT metric, but has several key differences. Instead of measuring the combined loss probability of a bidirectional transmission of a packet over a link in both directions, the Directional Airtime Metric measures the incoming loss rate and integrates the incoming linkspeed into the metric cost. There are multiple reasons for this decision: OLSRv2 defines the link metric as directional costs between nodes. Not all link layer implementations use acknowledgement mechanisms. Most link layer implementations who do use them use less airtime and a more robust modulation for the acknowledgement than the data transmission, which makes it more likely for the data transmission to be disrupted compared to the acknowledgement. Incoming packet loss and linkspeed can be measured locally, symmetric link loss would need an additional signaling TLV in the HELLO and would delay metric calculation by up to one HELLO interval. The Directional Airtime Metric does not integrate the packet size into the link cost. Doing so is not feasible in most link-state routing protocol implementations. The routing decision of most operation systems don't take packet size into account. Multiplying all link costs of a topology with the size of a data-plane packet would never change the dijkstra result anyways. The queue based packet loss estimator has been tested extensively in the OLSR.org ETX implementation, see . The output is the average of the packet loss over a configured time period.

The Directional Airtime Metric is calculated for each link set entry, as defined in section 7.1. The metric processes two kinds of data into the metric value, namely packet loss rate and link-speed. While the link-speed is taken from an external process, the current packet loss rate is calculated by keeping track of packet reception and packet loss events. Multiple incoming packet loss/reception events must be combined into a loss rate to get a smooth metric. Experiments with exponential weighted moving average (EWMA) lead to a highly fluctuating or a slow converging metric (or both). To get a smoother and more controllable metric result, this metric uses two fixed length queues to measure and average the incoming packet events, one queue for received packets and one for the estimated number of packets sent by the other side of the link. Because the rate of incoming packets is not uniform over time, the queue contains a number of counters, each representing a fixed time interval. Incoming packet loss and packet reception event are accumulated in the current queue element until a timer adds a new empty counter to both queues and remove the oldest counter from both. In addition to the packet loss stored in the queue, this metric uses a timer to detect a total link-loss. For every NHDP HELLO interval in which the metric received no packet from a neighbor, it scales the number of received packets in the queue based on the total time interval the queue represents compared to the total time of the lost HELLO intervals. The average packet loss ratio is calculated as the sum of the 'total packets' counters divided by the sum of the 'packets received' counters. This value is then divided through the current link-speed and then scaled into the range of metrics allowed for OLSRv2. The metric value is then used as L_in_metric of the Link Set (as defined in section 8.1. of ).

This specification defines the following parameters, which can be changed without making the metric outputs incomparable with each other: - Queue length for averaging packet loss. All received and lost packets within the queue are used to calculate the cost of the link. - interval in seconds between two metric recalculations as described in . This value SHOULD be smaller than a typical HELLO interval. - timeout factor for HELLO interval at which point a HELLO is definitely considered lost. The value must be a floating point number between 1.0 and 2.0, large enough to take the delay and jitter for message aggregation into account. - threshold in number of missing packets (based on received packet sequence numbers) at which point the router considers the neighbor has restarted. This parameter is only used for packet sequence number based loss estimation. This number MUST be larger than DAT_MAXIMUM_LOSS.

The proposed values of the protocol parameters are for Community Mesh Networks, which mostly use immobile mesh nodes. Using this metric for mobile networks might require shorter DAT_REFRESH_INTERVAL and/or DAT_MEMORY_LENGTH. := 64 := 1 := 1.2 := 256

This specification defines the following constants, which cannot be changed without making the metric outputs incomparable: - Fraction of the loss rate used in this routing metric. Loss rate will be between 0/DAT_MAXIMUM_LOSS and (DAT_MAXIMUM_LOSS-1)/DAT_MAXIMUM_LOSS: 4. - Minimal bit-rate in Bit/s used by this routing metric: 1024.

This specification extends the Link Set Tuples of the Interface Information Base, as defined in section 7.1, by the following additional elements for each link tuple when being used with this metric: is a QUEUE with DAT_MEMORY_LENGTH integer elements. Each entry contains the number of successfully received packets within an interval of DAT_REFRESH_INTERVAL. is a QUEUE with DAT_MEMORY_LENGTH integer elements. Each entry contains the estimated number of packets transmitted by the neighbor, based on the received packet sequence numbers within an interval of DAT_REFRESH_INTERVAL. is the time when the next hello will be expected. is the interval between two hello messages of the links neighbor as signaled by the INTERVAL_TIME TLV of NHDP messages . is the estimated number of lost hello messages from this neighbor, based on the value of the hello interval. is the current bitrate of incoming unicast traffic for this neighbor. Methods to obtain the value of L_DAT_rx_bitrate are out of the scope of this specification. Such methods may include static configuration via a configuration file or dynamic measurement through mechanisms described in a separate specification (e.g. ). Any Link tuple with L_status = HEARD or L_status = SYMMETRIC MUST have a specified value of L_DAT_rx_bitrate if it is to be used by this routing metric. When using packet sequence numbers to estimate the loss rate, the Link Set Tuples get another field: is the last received packet sequence number received from this link.

When generating a new tuple in the Link Set, as defined in section 12.5 bullet 3, the values of the elements specified in are set as follows: L_DAT_received := 0, ..., 0. The queue always has DAT_MEMORY_LENGTH elements. L_DAT_total := 0, ..., 0. The queue always has DAT_MEMORY_LENGTH elements. L_DAT_last_pkt_seqno := UNDEFINED (no earlier packet received). L_DAT_hello_time := EXPIRED (no earlier NHDP HELLO received). L_DAT_hello_interval := UNDEFINED (no earlier NHDP HELLO received). L_DAT_lost_hello_messages := 0 (no HELLO interval without packets).

For the purpose of this section, note the following definitions: "pkt_seqno" is defined as the packet sequence number of the received packet. "interval_time" is the time encoded in the INTERVAL_TIME message TLV of a received HELLO message.

An implementation of OLSRv2 using the metric specified by this document MUST include the following parts into its output: an INTERVAL_TIME message TLV in each HELLO message, as defined in section 4.3.2.

While this metric was designed for measuring the packet loss based on the packet sequence number, some implementations might not be able to add the packet sequence number to their output.

An implementation of OLSRv2, using the metric specified by this document with packet sequence based link loss, MUST include the following element into its output: an interface specific packet sequence number as defined in section 5.1 which is incremented by 1 for each outgoing packet on the interface. For each incoming packet, additional processing MUST be carried out after the packet messages have been processed as specified in and . packets without packet sequence number MUST NOT be processed in this way by this metric. The router MUST update the Link Set Tuple corresponding to the originator of the packet: If L_DAT_last_pkt_seqno = UNDEFINED, then: L_DAT_received[TAIL] := 1. L_DAT_total[TAIL] := 1. Otherwise: L_DAT_received[TAIL] := L_DAT_received[TAIL] + 1. diff := seq_diff(pkt_seqno, L_DAT_last_pkt_seqno). If diff > DAT_SEQNO_RESTART_DETECTION, then: diff := 1. L_DAT_total[TAIL] := L_DAT_total[TAIL] + diff. L_DAT_last_pkt_seqno := pkt_seqno. If L_DAT_hello_interval != UNDEFINED, then: L_DAT_hello_time := current time + (L_DAT_hello_interval * DAT_HELLO_TIMEOUT_FACTOR). L_DAT_lost_hello_messages := 0.

A metric might just use the incoming NHDP HELLO messages of a neighbor to calculate the link loss. Because this method uses fewer events to calculate the metric, the variance of the output will increase. It might be necessary to increase the value of DAT_MEMORY_LENGTH to compensate for this. For each incoming HELLO message, after it has been processed as defined in section 12, the Link Set Tuple as defined in section 7.1 corresponding to the incoming HELLO message must be updated. L_DAT_received[TAIL] := L_DAT_received[TAIL] + 1. L_DAT_total[TAIL] := L_DAT_total[TAIL] + 1. L_DAT_lost_hello_messages := 0.

Instead of using incoming packets or messages, the routing daemon can also use other sources to measure the link layer lossrate (e.g. ). To use a source like this with the DAT metric, the routing daemon has to add incoming total traffic (or the sum of received and lost traffic) and lost traffic to the queued elements in the extension of the Link Set Tuple defined in corresponding to originator of the traffic. The routing daemon should also set L_DAT_lost_hello_messages to zero every times new packages arrive.

For each incoming HELLO Message, after it has been processed as defined in section 12, the Link Set Tuple corresponding to the incoming HELLO message must be updated. Only HELLO messages with an INTERVAL_TIME message TLVs must be processed. L_DAT_hello_interval := interval_time.

When L_DAT_hello_time has timed out, the following step MUST be done: L_DAT_lost_hello_messages := L_DAT_lost_hello_messages + 1. L_DAT_hello_time := L_DAT_hello_time + L_DAT_hello_interval.

Once every DAT_REFRESH_INTERVAL, all L_in_metric values in all Link Set entries MUST be recalculated: sum_received := sum(L_DAT_total). sum_total := sum(L_DAT_received). If L_DAT_hello_interval != UNDEFINED and L_DAT_lost_hello_messages > 0, then: lost_time_proportion := L_DAT_hello_interval * L_DAT_lost_hello_messages / DAT_MEMORY_LENGTH. sum_received := sum_received * MAX ( 0, 1 - lost_time_proportion); If sum_received < 1, then: L_in_metric := MAXIMUM_METRIC, as defined in section 5.6.1. Otherwise: loss := sum_total / sum_received. If loss > DAT_MAXIMUM_LOSS, then: loss := DAT_MAXIMUM_LOSS. bitrate := L_DAT_rx_bitrate. If bitrate < DAT_MINIMUM_BITRATE, then: bitrate := DAT_MINIMUM_BITRATE. L_in_metric := (2^24 / DAT_MAXIMUM_LOSS) * loss / (bitrate / DAT_MINIMUM_BITRATE). remove(L_DAT_total) add(L_DAT_total, 0) remove(L_DAT_received) add(L_DAT_received, 0)

This document contains no actions for IANA.

Artificial manipulation of metrics values can drastically alter network performance. In particular, advertising a higher L_in_metric value may decrease the amount of incoming traffic, while advertising lower L_in_metric may increase the amount of incoming traffic. By artificially increasing or decreasing the L_in_metric values it advertises, a rogue router may thus attract or repulse data traffic. A rogue router may then potentially degrade data throughput by not forwarding data as it should or redirecting traffic into routing loops or bad links. An attacker might also inject packets with incorrect packet level sequence numbers, pretending to be somebody else. This attack could be prevented by the true originator of the RFC5444 packets by adding a ICV Packet TLV and TIMESTAMP Packet TLV to each packet. This allows the receiver to drop all incoming packets which have a forged packet source, both packets generated by the attacker or replayed packets.

The authors would like to acknowledge the network administrators from Freifunk Berlin and Funkfeuer Vienna for endless hours of testing and suggestions to improve the quality of the original ETX metric for the OLSR.org routing daemon. This effort/activity is supported by the European Community Framework Program 7 within the Future Internet Research and Experimentation Initiative (FIRE), Community Networks Testbed for the Future Internet (), contract FP7-288535. The authors would like to gratefully acknowledge the following people for intense technical discussions, early reviews and comments on the specification and its components (listed alphabetically): Teco Boot (Infinity Networks), Juliusz Chroboczek (PPS, University of Paris 7), Thomas Clausen, Christopher Dearlove (BAE Systems Advanced Technology Centre), Ulrich Herberg (Fujitsu Laboratories of America), Markus Kittenberger (Funkfeuer Vienna), Joseph Macker (Naval Research Laboratory) and Stan Ratliff (Cisco Systems).

Harvard University Project Hipercom, INRIA Rocquencourt, France Project Hipercom, INRIA Rocquencourt, France Ecole Polytechnique, France BAE Systems Naval Research Laboratory INRIA Rocquencourt Ecole Polytechnique, France BAE Systems Ecole Polytechnique, France BAE Systems Naval Research Laboratory Fujitsu Laboratories of America Ecole Polytechnique, France Ecole Polytechnique, France Project Hipercom, INRIA Rocquencourt, France BAE Systems Cisco Cisco Cisco NetApp Broadcom

The Funkfeuer and Freifunk networks are OLSR-based or B.A.T.M.A.N. based wireless community networks with hundreds of routers in permanent operation. The Vienna Funkfeuer network in Austria, for instance, consists of 400 routers (around 600 routes) covering the whole city of Vienna and beyond, spanning roughly 40km in diameter. It has been in operation since 2003 and supplies its users with Internet access. A particularity of the Vienna Funkfeuer network is that it manages to provide Internet access through a city wide, large scale Wi-Fi mesh network, with just a single Internet uplink. Operational experience of the OLSR project with these networks have revealed that the use of hop-count as routing metric leads to unsatisfactory network performance. Experiments with the ETX metric were therefore undertaken in parallel in the Berlin Freifunk network as well as in the Vienna Funkfeuer network in 2004, and found satisfactory, i.e., sufficiently easy to implement and providing sufficiently good performance. This metric has now been in operational use in these networks for several years. The ETX metric of a link is the estimated number of transmissions required to successfully send a packet (each packet equal to or smaller than MTU) over that link, until a link layer acknowledgement is received. The ETX metric is additive, i.e., the ETX metric of a path is the sum of the ETX metrics for each link on this path. While the ETX metric delivers a reasonable performance, it doesn't handle well networks with heterogeneous links that have different bitrates. Since every wireless link, when using ETX metric, is characterized only by its packet loss ratio, the ETX metric prefers long-ranged links with low bitrate (with low loss ratios) over short-ranged links with high bitrate (with higher but reasonable loss ratios). Such conditions, when they occur, can degrade the performance of a network considerably by not taking advantage of higher capacity links. Because of this the OLSR.org project has implemented the Directional Airtime Metric for OLSRv2, which has been inspired by the Estimated Travel Time (ETT) metric . This metric uses an unidirectional packet loss, but also takes the bitrate into account to create a more accurate description of the relative costs or capabilities of mesh links.