Re: [hybi] sample implementation of my most recent proposal posted

Willy Tarreau <w@1wt.eu> Tue, 31 August 2010 18:15 UTC

Date: Tue, 31 Aug 2010 20:15:36 +0200
From: Willy Tarreau <w@1wt.eu>
To: John Tamplin <jat@google.com>
Message-ID: <20100831181536.GB3272@1wt.eu>
References: <AANLkTin2xesaAgrj1zPUJVspiurhg6w3oNA6ZwWNvEHv@mail.gmail.com> <AANLkTikbj9SzZZTWpDj77-jfoT3MJ9eTKYXDwHR2ykGZ@mail.gmail.com> <AANLkTi=BB=638Sw8ZmVFdDrmALHCMncyCgLcRcuDVWcL@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <AANLkTi=BB=638Sw8ZmVFdDrmALHCMncyCgLcRcuDVWcL@mail.gmail.com>
User-Agent: Mutt/1.4.2.3i
Cc: Hybi <hybi@ietf.org>
Subject: Re: [hybi] sample implementation of my most recent proposal posted
Precedence: list

On Tue, Aug 31, 2010 at 01:45:35PM -0400, John Tamplin wrote:
> On Tue, Aug 31, 2010 at 1:19 PM, Pieter Hintjens <ph@imatix.com> wrote:
> > If the use case for fragmentation is streaming of arbitrarily long
> > messages whose size is unknown upfront (which is the only real use
> > case that's been cited afair, with reference to HTTP chunking)
> 
> The other main use case is for multiplexing, where you don't want
> other logical channels blocked behind one sending a very large frame,
> so you need to be able to fragment that frame.  Certainly, you could
> define fragmentation within the mux payload, but it seems cleaner if
> it can just reuse the fragmentation support defined in the base
> protocol.

That's one point where I agree with what was said by Shelby, we should
not confuse fragmentation of the connection stream caused by multiplexed
data streams and fragmentation of messages caused by unknown (or too
long) message lengths.

I propose that we use the word "fragmentation" when we're talking about
mixed streams and "chunking" when we're talking about a known-sized chunk
of a possibly unknown-sized message (as we do in HTTP).

Thus, a "fragment" is the smallest part of a connection stream, and a
"chunk" is the smallest part of a message. As long as we don't support
multiplexing, both are probably identical on the wire, but I suspect
this can change with multiplexing (fragments then being able to transport
parts of chunks).

To ease understanding, one might want to think about what HTTP chunks
look like at the IP level. You'll see interlaced TCP segments related
to various streams, carrying portions of message chunks. BTW, maybe the
word "segment" should be used instead of "fragment" ?

We should also be careful not to reproduce the issues of IP vs
fragmentation, where it is easy to make the receiver reserve large
amounts of memory while waiting for a fragment that never comes. This
becomes especially important with multiplexing, because I suspect that
receivers will refresh their timeouts as long as at least one stream
talks on top of a connection.

Regards,
Willy

Re: [hybi] sample implementation of my most recen… Pieter Hintjens
[hybi] sample implementation of my most recent pr… John Tamplin
Re: [hybi] sample implementation of my most recen… Greg Wilkins
Re: [hybi] sample implementation of my most recen… John Tamplin
Re: [hybi] sample implementation of my most recen… Hector Santos
Re: [hybi] sample implementation of my most recen… John Tamplin
Re: [hybi] sample implementation of my most recen… Shelby Moore
Re: [hybi] sample implementation of my most recen… John Tamplin
Re: [hybi] sample implementation of my most recen… Willy Tarreau
Re: [hybi] sample implementation of my most recen… Greg Wilkins
Re: [hybi] sample implementation of my most recen… John Tamplin
Re: [hybi] sample implementation of my most recen… Willy Tarreau