Re: [hybi] BWTP Proposal - Bidirection Web Transfer Protocol

[Jamie Lokier <jamie at shareable.org>]

Greg Wilkins wrote:
> So if you have traffic that is substantially one way, you end up
> sending lots of empty messages the other direction to keep things
> balanced.

It's not immediately obvious, but BWTP will send empty messages in the
other direction anyway.

They are called TCP ACKs, and you'd be surprised how little difference
a few "200 OK" lines makes per ACK, if they are piggybacked properly.

(Of course they aren't always piggybacked properly.)

I do like the idea of responses being optional though.  It's just
nicer, when you are layering another protocol on top of the messaging
service anyway - and the other protocol probably has it's _own_ form
of replies.

In my own bidirectional-HTTP-like implementation, the server always
generates a response, but each request has a field indicating if the
response should be sent or not.  (The same field is used to coordinate
out-of-order responses in fact).

> This degenerates to more-or-less long polling - which we can do
> today.

Not really.  With long polling, you need two sockets: one to send
messages, and one to receive them on.  Bidirectional messaging is an
improvement already.

> I guess there could be an incremental improvement in HTTP long
> polling by sending a full pipeline of requests and requiring the
> server to examine that incoming pipeline for messages, but there
> would be queueing delays and not all intermediaries can handle
> pipelines anyway.

I'm curious about that.  What known issues are there with
intermediaries and pipelines?  I'm aware of lots of problems with
origin servers and pipelines, but not intermediaries.

I guess known issues should go in the Comet best practices document?

> Finally, for the sake of debugging, it would be good if a BWTP
> protocol packet can be identified as such by inspection without
> state.

That makes good sense, I like that.

> But if you think that a pure RFC2616 like approach is worthwhile,
> then I'd certainly review and contribute to an alternate alternate
> proposal.

There are several problems and limitations with HTTP and pipelined
HTTP that this has a chance to fix, not least what to do on errors,
and what to do when a connection has been idle for a while.

Here's a couple of RFC2616 issues which are, imho, best avoided when
reinventing it bidirectionally:

RFC2616 specifies that under certain conditions, a connection must be
terminated _abruptly_ when an error response is sent to certain
requests like POST, to stop a potentially large request from being
sent whole.  Any pipelined requests after it will be lost, but you
cannot know which ones the server received and started processing.

You don't really want that here: it's much better that there's a way
to halt one request without breaking the connection for other requests.

When sending a request to an RFC2616 idle connection (say after 30
seconds), you don't know if the server has decided to close it or not.
So on sending, you might get a connection error, and then you don't
know if your request was processed or not, and whether it's safe to
repeated it.  There are other reasons why you might get the same
connection error, such as network faults and NAT timeouts.

For that reason, even persistent HTTP cannot be used for many requests
safely and automatically, so clients don't.  E.g. for POST.  Let alone
pipelined HTTP, which has the same problem in more cases.

You have, thankfully, already addressed the orderly shutdown problem.
Real HTTP servers use a lingering half-closed socket, to ensure the
last response will actually reach the client if the client pipelined
another request.  This is necessary for reliable behaviour (due to a
TCP quirk), yet not mentioned in RFC2616.

All messages over BWTP would all fall into the category of not being
safe to repeat automatically by the BWTP implementation.  Now, because
BWTP is intended for "TCP-like" applications, they will already be
aware of the possibility of broken connections and such things, and
have their own method of recovering, so it's not such a big problem.

But that means you cannot simply layer standard _HTTP_ requests over
BWTP safely.  Only application-specific protocols which know what to
expect can use it, even though they have access to HTTP-style message
metadata.

It _is_ possible to layer standard HTTP over BWTP safely, and to avoid
the same problems in BWTP-aware applications (making the applications
simpler).  But not by taking RFC2616 and simply making it bidirectional.

> > One virtue of staying closer to HTTP is that one can easily imagine how
> > existing AJAX APIs and code could be adjusted to use the BWTP socket
> > easily. For that reason, I am leaning towards the belief that BWTP
> > should try to be a superset of unidirectional HTTP.

I like that idea a lot (I'll help make it possible), but I also like
the idea of having the option to transport simple non-HTTP messages too.

See above about why you have to be careful about layering HTTP
messages automatically over anything else that dabbles in persistence
and pipelining.  There are assumptions about what behaviour to expect,
and what requests are safe to repeat automatically in the protocol
implementation, and when to do it.  AJAX apps expect to call
XmlHttpRequest and not have to do things like recover from
broken/timed-out connections - they expect the protocol implementation
to sort it out, but in a safe way.

-- Jamie