On Fri, Aug 20, 2010 at 5:07 AM, gustav trede <gustav.trede@gmail.com>wrote:

> Where is the technical sound reasoning for the 64 bit frame length in the
> first place ?.
> There is a lot of talk that the expected frame lengths are to be small, so
> why have 64 bit at all ?, it just penalizes whenever stuff happens to not
> fit in the tiny length, at least make that penalty reasonable - 32 bits
> should be ok, the few with larger needs can use multi frame in 4gig blocksm
> they for sure dont notice that overhead - so let them have it instead of the
> majority that is the smaller frames.
>

There were significant discussions about wanting to be able to write a
header and call sendfile() to write the contents of a large file [1], rather
than having to stay involved and write blocks repeatedly.  Granted, this is
not useful today given the JS API where the entire message has to be
buffered, but: 1) there will be non-JS WebSocket clients that could easily
support a streaming API, 2) the framing protocol has to be defined now in
such a way that implementations of the initial spec can interoperate with
ones implementing a future extension of this spec.  As the frame size is
something that has to be understood to even understand frame boundaries,
that means the way to obtain the frame size, even for frames used by future
extensions, must be present in the base protocol.

Yes, it is true that I expect a significant percentage of the frames of real
WebSocket apps to fit in the small frame size (especially when compressed),
and that there is a real desire to be able to exchange WebSocket frames at
3G FACH power levels rather than having to go up to DACH (see [2] for some
details), but we still need to be able to accommodate larger frames.

When I posted an message summarizing the various positions on fragmentation
and lengths [3], there were no objections to the general principles - only
quibbles about the details.  That and the discussions referenced in [1]
suggests any fixed length shorter than 64-bits would not be acceptable to
large numbers of people on the list, which is why I relented and suggested a
large enough frame size that any message could be sent unfragmented if
desired.

Its fundamentally broken to specify a limit will break at random lower
> limits in the real world, we don't offer a path-mtu so its a guessing game.
>

Sure, lower layers will fragment the large frames, but that happens with
TCP/IP now.  Is SMTP broken because it doesn't try and avoid fragmentation
at the network layer and instead just streams TCP data?


> It would be prudent to have a standard way for a peer to know the upper
> frame length limit it can safely use,
> currently we cant detect if a used frame length is too large, we don't know
> if that's the reason the connection closed when we did send the super huge
> frame.
>

If we were to negotiate it, it would not be at the framing layer but at the
connection handshake.  As far as signalling an error, we haven't really
nailed down the control frames and I would not be opposed to having an
optional error indication in the close control frame which could indicate
why it was dropped.


> A standard header stating the peer upper frame length limit during the
> existing handshake could perhaps solve this problem ?.
>

Correct.


> IPV6 uses 32bit -1 as the largest jumbo frame size, why do we need more ? -
> that argument should technically justify the overhead it brings,
> so please have something better then just repeating "other layers overhead
> justifies us to spew more overhead on top",
>

WebSockets is intended to be a minimal message-oriented protocol on top of
TCP.  TCP itself has no notion of maximum message size -- you can just write
arbitrary number of bytes, regardless of the MTU of the underlying network,
and they get there.  I am not sure why you see that WebSockets should care
more about the underlying MTU than TCP does.


> what if every layer spec was designed by that motto ?!,
> Also using percentage comparisons vs protocols below us as tool to make it
> look better then it is in absolute terms is just a bad marketing trick.
> Peoples expected incompetence is not a much better argument (.
>

I think percentage does make sense because if you are sending 4G of data,
does a wasted 4 bytes in the length field mean anything?  No.

Initially, v76 proposed a variable-length length field for binary frames and
sentinel framing for text frames [4].  There was near universal agreement on
this list that we should move to a single length-based framing for both text
and data, and there was enough support against the variable-length length
field that Ian Hickson switched to a fixed 64-bit length field in v00 [5].
 This was also based on discussions where the frame length was proposed as
16 bits, requiring larger messages to be fragmented, and there was
significant opposition on the grounds that it required simple senders to
have to implement fragmentation and inhibited use of sendfile.  Greg also
noted that the initial Jetty implementation used a signed 32-bit length
internally and failed in a real-world case, so it seems more than 31 bits
are required today, not to mention what will become common during the life
of this protocol [6].

Personally, I don't find a fixed 64-bit length acceptable because it imposes
a significant penalty on small frames.  Thus, the proposal of having two
sizes of lengths: a small one where we care about preserving small frames
for small messages, and a larger one that will meet every need.

I am less concerned about choosing between a v76-style variable length and
as in my most recent proposal -- I prefer what I proposed for the reasons
listed in this thread, but I would not object strongly to switching.

What is the best way to decide between these?   Does anyone feel very
strongly between 7/63 bit lengths and {1-9}*7 bit lengths?  Should we take a
vote?


Related threads:

   - [1] Discussions about length choices
      - http://www.ietf.org/mail-archive/web/hybi/current/msg02974.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg02977.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg03025.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg03039.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg03051.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg03061.html
      - http://www.ietf.org/mail-archive/web/hybi/current/msg03064.html
      -
      - [2] http://xmpp.org/extensions/inbox/mobile.html#sect-id129052
   - [3] http://www.ietf.org/mail-archive/web/hybi/current/msg03069.html
   - [4] http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76
   - [5] http://www.whatwg.org/specs/web-socket-protocol/
   - [6] http://www.ietf.org/mail-archive/web/hybi/current/msg03105.html

-- 
John A. Tamplin
Software Engineer (GWT), Google