Dear All,

We've mentioned ICMP Source Quench a few times on the mailing list, and people asked about ICMP Source Quench and about ECN at the BOF. Carl and I weren't "there" for either ICMP Source Quench or ECN experience, but we're attempting to capture the collective wisdom (based on RFCs, etc).

Please don't read the following as a definitive statement - it's a question that needs to be answered in any TRIGTRAN work going forward, and we're interested in other understandings of the collective wisdom.

PLEASE SEND REBUTTALS AND CORRECTIONS!

Thanks,

Spencer, for Spencer and Carl

--------------------------

At the IETF 55 TRIGTRAN BoF, Sally Floyd presented a number of questions for TRIGTRAN. One of the most relevant was "ICMP Source Quench failed. P-MTU Discovery failed. Why will TRIGTRAN be different?". This question needs to be answered. Our crack at an answer follows.

ICMP Source Quench
------------------
RFC 792 describes the Internet Control Management Protocol (ICMP). ICMP includes a message type called "Source Quench" (Type 4). Source Quench was intended to provide "back pressure" when a gateway discards an incoming IP datagram because no buffers are available. The message is sent to the Source IP Address carried in the discarded IP datagram. Conceptually, a host receiving an ICMP Source Quench message would slow down its sending rate until it stopped receiving ICMP Source Quench messages, and then gradually increase its sending rate.

The original specification did not provide quantitative guidance on HOW MUCH to slow down. RFC 1016 proposed a formula Source Quench Induced Delay ("SQuID"), but this RFC was published six years after RFC 792, and defined itself as a "crazy idea". A better characterization might have been "embryonic", reflecting an unsophisticated awareness of congestion - TCP didn't include Slow Start/Congestion Avoidance until a couple of years later. 

The original specification allowed gateways to generate an ICMP Source Quench for every datagram dropped, but did not require gateways to send them at all. RFC 1009 ("Requirements for Internet Gateways") required gateways to include the capability to send rate-limited ICMP Source Quench messages, but when it was updated as RFC 1812 ("Requirements for IP Version 4 Routers"), this requirement was dropped in favor of deprecating ICMP Source Quench ("Gateways SHOULD NOT"). The reasons given for this about-face included:

- ICMP Source Quench affected only packets sent from the host generating the "over the top" packet, so did not provide a fair mechanism for hosts sharing overcommitted network paths, and 

- ICMP Source Quench added (reverse-direction) packets to the network during congestion events, and used router memory and processing power to construct and send ICMP Source Quenches during congestion events.

The overwhelming problem with ICMP Source Quench wasn't that it required gateways to send a "trigger" to hosts - it had problems with unfairness and inefficiency. Since the specifications omitted critical details and didn't require this functionality, hosts were forced to add end-to-end congestion avoidance at the transport layer, anyway.

This experience isn't terrically relevant to TRIGTRAN, because recommendations have shifted from MAY to SHOULD NOT - hardly an overwhelming push to deployment!

Path Maximum Transmission Unit Discovery
----------------------------------------

RFC 791 ("Internet Protocol") allows IP packets of up to 65,535 octets, but subnetworks typically don't support frames with a payload this large. A host can know the Maximum Transmission Unit size for its local subnetwork, but can't be sure that a path across multiple subnetworks will support a larger MTU without IP fragmentation. RFC 1122 ("Requirements for Internet Hosts -- Communication Layers") specified that IP implementations "SHOULD" use an MTU of 576 or less to communicate with hosts on a different network, unless the implementation "knew" that the path supported larger MTUs.

RFC 1063 ("IP MTU Discovery Options") and its successor RFC 1191 ("Path MTU Discovery") described a mechanism to gain this knowledge. An IP implementation wishing to use large MTUs sent a packet of the desired size into the network with the "Don't Fragment" bit set. If the packet encountered a subnetwork that didn't support the desired MTU size, the gateway for that link discarded the packet, and reported an ICMP ICMP Destination Unreachable error with a code meaning "fragmentation needed and DF set", (also described as "Datagram Too Big"), and an indication of the bottleneck MTU size, stored in a previously-unused ICMP header field. To avoid requiring a "flag day" for P-MTU discovery, hosts were required to accept "Datagram Too Big" errors that didn't include the bottleneck MTU size, and to either fall back to 576 bytes or search with a new, lower P-MTU "guess", thus accommodating gateways that hadn't been updated.

As P-MTU Discovery was deployed, another "discovery" happened. As described in RFC 1435 ("IESG Advice from Experience with Path MTU Discovery"), "some vendors have added to their routers the ability to disable ICMP messages generated by the router". The effect was that of a "black hole" - the router dropped the "too big" datagram, but did not send any notification to the sending host that this was happening. Further deployment experience led to RFC 2923 ("TCP Problems with Path MTU Discovery"), pointing out that "black holing" was still happening, and that routers were configured this way for a variety of reasons, including a misguided attempt to shut down ICMP messages crossing firewalled administrative domains. Procedures for hosts encountering "black holes" were described, but guessing too high on a "black-holed" path still leads to delays of several seconds as the host TCP implementation detects failure using timeouts before it "falls back" to 576 bytes.

The Internet experience with P-MTU Discovery is more relevant to TRIGTRAN if TRIGTRAN considers ICMP messages as a trigger signal mechanism, although the "black holing" problem is less severe because TRIGTRAN triggers are advisory in nature. End-to-end mechanisms will lead to the the same result after some delay. 

The history of P-MTU Discovery is useful as a reminder that TRIGTRAN triggers must traverse firewalls, no matter what mechanism is defined to transport them.
_______________________________________________
Trigtran mailing list
Trigtran@ietf.org
https://www1.ietf.org/mailman/listinfo/trigtran