Advanced Stream and Sampling Framework for IPPM

There are several areas where measurement methodology definition and test result interpretation will benefit from an increased understanding of the stream characteristics and the (possibly unknown) network condition that influence the measured metrics. Network treatment depends on the fullest extent on the "packet of Type-P" definition in , and has for some time. State is often maintained on the per-flow basis at various points in the network, where "flows" are determined by IP and other layers. Significant treatment differences occur with the simplest of Type-P parameters: packet length. Payload content optimization (compression or format conversion) in intermediate segments. This breaks the convention of payload correspondence when correlating measurements made at different points in a path. Packet history (instantaneous or recent test rate or inactivity, also for non-test traffic) profoundly influences measured performance, in addition to all the Type-P parameters described in . Access technology may change during testing. A range of transfer capacities and access methods may be encountered during a test session. When different interfaces are used, the host seeking access will be aware of the technology change which differentiates this form of path change from other changes in network state. Section 14 of treats the possibility that a host may have more than one attachment to the network, and also that assessment of the measurement path (route) is valid for some length of time (in Section 5 and Section 7 of ). Here we combine these two considerations under the assumption that changes may be more frequent and possibly have greater consequences on performance metrics. Paths including links or nodes with time-slotted service opportunities represent several challenges to measurement (when service time period is appreciable): Random/unbiased sampling is not possible beyond one such link in the path. The above encourages a segmented approach to end to end measurement, as described in for Network Characterization (as defined in ) to understand the full range of delay and delay variation on the path. Alternatively, if application performance estimation is the goal (also defined in ), then a stream with un-biased or known-bias properties may be sufficient. Multi-modal delay variation makes central statistics unimportant, others must be used instead. Each of these topics is treated in detail below.

We recommend two Type-P parameters to be added to the factors which have impact on network performance measurements, namely packet length and payload type. Carefully choosing these parameters can improve measurement methodologies in their continuity and repeatability when deployed in reactive networks.

Many instances of network characterization using IPPM metrics have relied on a single test packet length. When testing to assess application performance or an aggregate of traffic, benchmarking methods have used a range of fixed lengths and frequently augmented fixed size tests with a mixture of sizes, or IMIX as described in . Test packet length influences delay measurements, in that the IPPM one-way delay metric includes serialization time in its first-bit to last bit time stamping requirements. However, different sizes can have a larger effect on link delay and link delay variation than serialization would explain alone. This effect can be non-linear and change instantaneous or future network performance. Repeatability is a main measurement methodology goal as stated in section 6.2 of . To eliminate packet length as a potential measurement uncertainty factor, successive measurements must use identical traffic patterns. In practice a combination of random payload and random start time can yield representative results as illustrated in .

The aim for efficient network resource use has resulted in a series of "smart" networks to deploy server-only or client-server lossless or lossy payload compression techniques on some links or paths. These optimizers attempt to compress high-volume traffic in order to reduce network load. Files are analyzed by application-layer parsers and parts (like comments) might be dropped. Although typically acting on HTTP or JPEG files, compression might affect measurement packets, too. In particular measurement packets are qualified for efficient compression when they use standard plain-text payload. IPPM-conforming measurements should add packet payload content as a Type-P parameter which can help to improve measurement determinism. Some packet payloads are more susceptible to compression than others, but optimizers in the measurement path can be out ruled by using incompressible packet payload. This payload content could be either generated by a random device or by using part of a compressed file (e.g., a part of a ZIP compressed archive).

Recent packet history and instantaneous data rate influence measurement results for reactive links supporting on-demand capacity allocation. Measurement uncertainty may be reduced by knowledge of measurement packet history and total host load. Additionally, small changes in history, e.g., because of lost packets along the path, can be the cause of large performance variations. For instance delay in reactive 3G networks like High Speed Packet Access (HSPA) depends to a large extent on the test traffic data rate. The reactive resource allocation strategy in these networks affects the uplink direction in particular. Small changes in data rate can be the reason of more than 200% increase in delay, depending on the specific packet size.

discussed the scenario of multi-homed hosts. If hosts become aware of access technology changes (e.g., because of IP address changes or lower layer information) and make this information available, measurement methodologies can use this information to improve measurement representativeness and relevance. However, today's various access network technologies can present the same physical interface to the host. A host may or may not become aware when its access technology changes on such an interface. Measurements for networks which support on-demand capacity allocation are therefore challenging in that it is difficult to differentiate between access technology changes (e.g., because of mobility) and reactive network behavior (e.g., because of data rate change).

Time-Slotted operation of network entities - interfaces, routers or links - in a network path is a particular challenge for measurements, especially if the time slot period is substantial. The central observation as an extension to Poisson stream sampling in is that the first such time-slotted component cancels unbiased measurement stream sampling. In the worst case, time-slotted operation converts an unbiased, random measurement packet stream into a periodic packet stream. Being heavily biased, these packets may interact with periodic network behavior of subsequent time-slotted network entities. Practical measurements confirm that such interference limits delay measurement variation to a sub-set of theoretical value range. Measurement samples for such cases can aggregate on artificial limits, generating multi-modal distributions as demonstrated in . In this context, the desirable measurement sample statistics differentiate between multi-modal delay distributions caused by reactive network behavior and the ones due to time-slotted interference. The amount of measurement bias is determined by the relative offset between allocated time-slots in subsequent network entities, delay variation in these networks, and other sources of variation. Measurement results might change over time, depending on how accurately the sending host, receiving host, and time-slotted components in the measurement path are synchronized to each other and to global time. If network segments maintain flow state, flow parameter change or flow re-allocations can cause substantial variation in measurement results. Measurement methodology selection for time-slotted paths depends to a large extent on the respective viewpoint. End-to-end metrics can provide accurate measurement results for short-term sessions and low likelihood of flow state modifications. Applications or services which aim at approximating network performance for a short time interval (in the order of minutes) and expect stable network conditions should therefore prefer end-to-end metrics. Here stable network conditions refer to any kind of global knowledge concerning measurement path flow state and flow parameters. However, if long-term forecast of time-slotted network performance is the main measurement goal, a segmented approach relying on hop-by-hop metrics is preferred. Re-generating unbiased measurement traffic at any hop can help to unleash the true range of network performance for all network segments.