Re: [tcpm] TCP tuning

[rick jones <perfgeek at mac.com>]

On Feb 3, 2010, at 7:49 PM, Joe Touch wrote:
> rick jones wrote:
> > Modulo slow-start after idle.
>
> I've already cited the work that explains that this doesn't actually
> happen on web exchanges exactly because the client request is small
> (doesn't matter that its side was idle), and the client request causes
> the server to think it wasn't idle either. That's in the ToN paper I  
> cited.
>
> Yes, if/when that's fixed, it'll be an issue. Not right now, though.

I'd not looked far ahead enough in my unread mail queue when I sent my  
message.  Still...  I have run some netperf TCP_RR tests between my  
home system:

Darwin rick-2 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01  
PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386

and netperf.org:

Linux www.netperf.org 2.6.31-15-server #50-Ubuntu SMP Tue Nov 10  
15:50:36 UTC 2009 x86_64 GNU/Linux

To avoid any issues with TSO clouding the trace, I disabled TSO and  
GSO on netperf.org.

On my home system I did ./configure --enable-intervals then built  
netperf, and then while running tcpdump on netperf.org executed:

src/netperf -l 300 -p 12345 -t TCP_RR -H netperf.org -b 1 -w 1s -- -r  
128,16384 -P 23456
TCP REQUEST/RESPONSE TEST from (null) (0.0.0.0) port 23456 AF_INET to  
netperf.org (192.6.38.59) port 23456 AF_INET : interval
Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate
bytes  Bytes  bytes    bytes   secs.    per sec

131070 262140 128      16384   300.00      1.00
16384  87380

Netperf will send a 128 byte request to netperf.org, which will  
respond immediately.  Netperf then waits one second before sending the  
next request etc etc...

The raw and "cooked" traces from that can be seen at:

ftp://ftp.netperf.org/traces

There one can also see "nothinktime" raw and cooked traces for:

[rick-2:~/netperf2_work] perfgeek% src/netperf -l 30 -p 12345 -t  
TCP_RR -H netperf.org  -- -r 128,16384 -P 23456
TCP REQUEST/RESPONSE TEST from (null) (0.0.0.0) port 23456 AF_INET to  
netperf.org (192.6.38.59) port 23456 AF_INET : interval
tcpLocal /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate
bytes  Bytes  bytes    bytes   secs.    per sec

131070 262140 128      16384   30.00       7.60
16384  87380

It certainly appears that *something* is holding-back the server in  
the "thinktime" case - I don't know if it is slowstart after idle, but  
it is something that does not appear to be holding-back the server in  
the "nothinktime" case.  Hmm... while writing this message, I have  
remembered there is a sysctl in linux to control slow-start after  
idle, so there is also a third trace, one with thinktime, and  
net.ipv4.tcp_slow_start_after_idle = 0.  From the look of that trace,  
and how it compares with the other two, I think that Linux 2.6.31 at  
least is not actually having the arriving request cause it to think it  
wasn't idle?

happy benchmarking,

rick jones

PS - I suppose if I'd been thinking more clearly, I'd have also  
included --enable-historgram in the ./configure command and been able  
to give a histogram of the response times (adding a -v 2 option to the  
global command line options), but I suppose that can be guesstimated  
from the traces.

http://homepage.mac.com/perfgeek