Re: [hybi] websocket test suite

Greg Wilkins <gregw@intalio.com> Fri, 29 July 2011 03:15 UTC

Return-Path: <gregw@intalio.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 09D5D21F8877 for <hybi@ietfa.amsl.com>; Thu, 28 Jul 2011 20:15:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.293
X-Spam-Level:
X-Spam-Status: No, score=-2.293 tagged_above=-999 required=5 tests=[AWL=-0.516, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_14=0.6, J_CHICKENPOX_46=0.6, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pGLoAMj3785Y for <hybi@ietfa.amsl.com>; Thu, 28 Jul 2011 20:15:19 -0700 (PDT)
Received: from mail-vx0-f172.google.com (mail-vx0-f172.google.com [209.85.220.172]) by ietfa.amsl.com (Postfix) with ESMTP id 6A0C421F8865 for <hybi@ietf.org>; Thu, 28 Jul 2011 20:15:19 -0700 (PDT)
Received: by vxi40 with SMTP id 40so3007993vxi.31 for <hybi@ietf.org>; Thu, 28 Jul 2011 20:15:18 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.52.88.2 with SMTP id bc2mr830074vdb.162.1311909318822; Thu, 28 Jul 2011 20:15:18 -0700 (PDT)
Received: by 10.52.114.228 with HTTP; Thu, 28 Jul 2011 20:15:18 -0700 (PDT)
In-Reply-To: <CABDh0KkxLLU1onDbiya_n2BAmyUBPkM1LS1PDn2_CHgnOWtpJg@mail.gmail.com>
References: <CAH_y2NFBNU7fiWWBoc2NNG0irjkOp7JAwDuNcAj-M2nFrGpVaw@mail.gmail.com> <CABDh0KkxLLU1onDbiya_n2BAmyUBPkM1LS1PDn2_CHgnOWtpJg@mail.gmail.com>
Date: Fri, 29 Jul 2011 13:15:18 +1000
Message-ID: <CAH_y2NGV28EHRhPZpCFWJfjTcR0_BGCtYv2yj4p0JA+26hU9_w@mail.gmail.com>
From: Greg Wilkins <gregw@intalio.com>
To: Hybi <hybi@ietf.org>
Content-Type: text/plain; charset="ISO-8859-1"
Subject: Re: [hybi] websocket test suite
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Jul 2011 03:15:20 -0000

Here is a first cut at a list of tests for section 4 of the spec.  Are
these the kinds of tests people were thinking about?
If so, then I'm happy to go the next step of detail and start
providing more detail for each test.

Obviously some implementation are not going to be able to send some of
the invalid frames or force fragmentation, so some suites will have to
record some tests as skipped, or only testing in one direction etc.




4.2 Base Framing Tests
----------------------

* 7 bit length: send/receive frames of length 0-125
* 16 bit length: send/receive frames of length 126-128, 65536
* 63 bit length: send/receive frames of length 65537

* Non zero RSV: send frames with non zero reserved bits and no
extension. Verify connection is failed.

* Known Opcodes: send/receive frames with text,binary,ping,pong
opcodes (continuation & close tested separately).
* Unknown Opcodes: send frames with unknown opcodes 3-7, B-F. Verify
connection is failed.


4.3 Client to Server Masking
----------------------------

???


4.4 Fragmentation
-----------------

* 1 fragment message: send/receive message in 1 fragment
* 2 fragment message: send/receive message in 2 fragments
* 3 fragment message: send/receive message in 3 fragments
* Injected control: send/receive 2 fragment message with unsolicited
pong message between frames
* Fragmented control: send fragmented ping. Verify that connection is failed.
* Interleaved fragments: send a fragment with FIN set and Continuation
opcode. Verify that connection is failed.
* Handled control: send/receive 2 fragment message with ping message
between frames and last frame delayed. Verify pong is received before
last frame is sent.
* 7 bit fragment: send/receive message with initial/middle/final
fragments of sizes 0-125
* 16 bit fragment: send/receive message with initial/middle/final
fragments of sizes 126-128,65536
* 64 bit fragment: send/receive message with
initial/middle/finalfragments of size 65537


4.5.1 Close Frames
------------------
* Orderly Close: On an idle connection, send a close and receive a close.
* Busy Close: send a close to an endpoint that has sent the first
fragment of a message.
* Simultaneous Close: send a close on both ends of a connection.
* Message after close: attempt to send a message after a close. Verify
message is not received.
* Idle close: ??
* Error Close: ??


4.5.2 Ping
----------
* Empty ping: Send an empty ping, verify pong is received.
* Non empty ping: Send a non empty ping, verify pong is received.


4.5.3 Pong
----------
* Empty pong: Send an empty ping, verify pong is received and is empty.
* Non empty pong: Send a non empty ping, verify pong is received and
has matching content.
* Outstanding ping: Send two pings with different content, reply with
content for only the second. Verify connection stays open.
* Unsolicited pong: Send/recv unsolicited pong.


4.5.4 Data Frames
-----------------
* UTF-8 Text 0000-007f: Send/receive text message with characters in
code point range  U+0000 to U+007F
* UTF-8 Text 0080-07ff: Send/receive text message with characters in
code point range  U+0080 to U+07FF
* UTF-8 Text 0800-ffff: Send/receive text message with characters in
code point range  U+0800 to U+FFFF
* UTF-8 Text 010000-10ffff: Send/receive text message with characters
in code point range  U+010000 to U+10FFFF
* Illegal bytes UTF-8: Send message with invalid UTF-8 byte sequence.
Verify message is not received.
* Illegal codes UTF-8: Send message with invalid UTF-8 code points.
Verify message is not received.
* Binary frames: Send/receive message containing illegal UTF-8 bytes
sequences and code points as binary message.