Re: [rohc] SigComp torture tests

[Pekka Pessi <Pekka.Pessi at nokia.com>]

Hi Abbie,

>Thanks for the extra corner cases on the state create test.  They are
>well worth testing (how long did it take you to find 2 pieces of state
>that have the same first 6 bytes of hash? ;-) ).

That was surpringly fast. I first hashed a million different
states at 256, then started generating state2 at 266. I estimated
it would take something like two days but I found first collision
after 3 minutes.

>I think there is also another useful test covered by the way you've
>written the bytecode :-)

>	7.  Write the bytes of the identifier to the position specified
>in the STATE-FREE instruction after the STATE-FREE instruction has been
>run (and before END-MESSAGE).

OK.

>I have a few comments on the message listed as input (I think we might
>not need quite so many) but am perfectly happy with the mnemonic and
>bytecode.  

>> And the test messages:

>>    Compressed message:       Number of state items:     UDVM cycles:
>>         0x00                          0                      13
>This one doesn't do anything - do we need it?

Probably not.

>>         0x0400                        Decompression failure   
>Is there any reason why this is 0420 rather than 0405 which would test 1
>byte below the limit?                    

No particular reason. Off-by-one might be more useful test.

>>         0x0420                        Decompression failure   

>Is there any reason why this is 0420 rather than 0415 which would test 1
>byte beyond the limit?                    

Ditto.

>>         0x0406                        No                     23
>						change 'No' to '0'

>>         0x09                          1                      34
>>         0x0a                          0                      25
>>         0x0b                          1                      35

>As with the previous version of the test, 0x0a and 0x0b *almost* but
>don't quite (if you're being strict about it) cover point 3. 0x0a
>recreates an existing piece of state before freeing it; 0x0b creates,
>frees and recreates.  However, 0x1e07 and 0x1e14 below both completely
>cover point 3 so I'm not sure we need 0x0a and 0x0b.  What do you think?

Hm. Probably not needed.

>>         0x1a                          2                      36
>>         0x1e06                        2                      46

>0x1e06 does pretty much the same as 0x1a (just with 2 attempts to free
>the state).  However, it does make a good comparison with 0x1e07 so
>maybe we don't need 0x1a?

OK.

>>         0x1e07                        0                      47
>>         0x1e14                        0                      60
>>         0x1f14                        1                      70

>Similarly, 0x1f14 doesn't really show anything that isn't already shown
>so I don't think we need it.  What do you think?

Nope.

>None of these is a big deal, but given that the test can be covered by a
>smaller number of test messages I don't see any point in over
>complicating it.

I usually let quantity to compensate for the quality when
generating tests, but you have a point here.

>> I took the UDVM cycle count from my notoriously reliable
>> debugging output. Caveat emptor.

>Our implementation agrees with your cycle counts :-)

Excellent. At least, there are bug-compatible implementations... ;)

--Pekka

_______________________________________________
Rohc mailing list
Rohc at ietf.org
https://www1.ietf.org/mailman/listinfo/rohc