[ippm] FW: [MMUSIC] Re: [Sip] RE: [BEHAVE] Re: ICE deploymentdatabefore LCfor RFC

Ø I’d suggest that the only way to test ICE (or any other NAT-traversal implementation) is to actually run it on thousands of computers hosted

Ø behind real-world network connections and see how well it works.

I fully agree and to make this work we need some tools that folks like here in the IETF WGs can download and report back the results.

The requirements for such a NAT testing tool would make a nice I-D on the condition someone is willing to write the code and make it available on a web site, similar to most beta testing sites, say in this case DSL Reports - > Speed Tests.

Thanks, Henry

From: David Barrett [mailto:dbarrett at quinthar.com]
Sent: Sunday, July 29, 2007 12:00 PM
To: 'Adam Fisk'; 'Jiri Kuthan'
Cc: Henry Sinnreich; 'Jan Janak'; 'Juha Heinanen'; mmusic at ietf.org
Subject: RE: [MMUSIC] Re: [Sip] RE: [BEHAVE] Re: ICE deploymentdatabefore LC for RFC

From: adamfisk at gmail.com
Subject: Re: [MMUSIC] Re: [Sip] RE: [BEHAVE] Re: ICE deploymentdatabefore LC for RFC

This has me worried. We all know the frightening number of NAT configurations out there. Maybe I missed it, but is there a test matrix somewhere of NAT combinations and their success rates with ICE? If not, could we create one using one of the open source ICE implementations out there (mine's not quite ready)? I'd be happy to contribute my own router if we had a simple test program, maybe based on the work at Nokia.

I’m not sure that matrixing out connectivity through NAT models is the right approach. Every NAT has a hundred ways to be configured (and misconfigured), and generally at least two of them sit between most client pairs. The number of real-world permutations is astronomical, and it almost certainly won’t catch the weird cases that can and do happen with surprising frequency.

A recent example is the bizarre NAT I mentioned before that duplicates every packet sent 3-7 times (at the “It’s A Grind” coffeeshop on Polk St in San Francisco). I’ve also dealt with the Google WiFi network (in Union Square, San Francisco), which has the unusual property of only being able to receive UDP from exactly *one* remote host per port – so you can send from a UDP port to anywhere in the world just fine (with symmetric behavior), but you can only receive back from the first – all other responses are blocked.

These are just two that I’ve personally experienced and that come up from recent memory: I’ve seen even more unusual behavior I can’t even begin to explain in my users’ logs. I’d suggest that the only way to test ICE (or any other NAT-traversal implementation) is to actually run it on thousands of computers hosted behind real-world network connections and see how well it works.

I can’t overstress how unusual, slow, unpredictable, and unreliable the internet can get in the real world, and how making thing works in the real world is far more difficult than making it work on paper. The faster you embrace real-world, large-scale testing, the faster you have something that works outside the lab.

-david