Re: [bmwg] [answer to Bill Cerveny] RE: New Version Notification for draft-janovak-bmwg-ipflow-meth-00 [answer to Bill Cerveny]

[Jeff Byzek <byzek at cisco.com>]

Bill Cerveny wrote:
>>> Wouldn't a big consideration be the quality and accuracy of the flow
>>> data?  How would the accuracy of the generated flow be evaluated?  The
>>
>>> flow generating capabilities of a network device is certainly less
>>> useful if the device's flow accuracy degrades when it is transiting
>>> more traffic.
>>
>> I would consider this out of the scope since to check that the testing
>> would
>> have to involve some external collector and the way those handle data
>> is even more complex than what happens on the network devices when
>> just registering the traffic.
>> I will think about it yet, but based on the testing we have done I don't
>> quite see how to do it.
>
> Couldn't you do something like send a known set of network traffic
> (defined by number of packets, types of packets, packet sizes, etc)
> across the router?  If the collector's interpretation of of the flow
> data matches what was sent across the router, a statement could be
> made about flow accuracy. If the collector's interpretation of the
> flow data does not match what was sent across the router, then a
> statement of accuracy cannot be made.  If the collector is unable to
> keep up with the flow data, perhaps another way of verifying the
> accuracy of the flow may be recording the flow data and playing it
> back at a rate that a collector could understand.
>
> I just think that making statements about a router's flow handling and
> reporting capabilities aren't complete if it can't be verified that
> the flow data reported is accurate.

This gets into the area of functionality verification and not
measurement, which seems to contradict the charter of the group (but I
could be misinterpreting).

Beyond this, there are many different variables involved, all of which
may have varying significance for different parties and can affect
accuracy in different ways.  It seems overly arduous to attempt to
identify a set (or multiple sets) of parameters that will satisfy all
(or even most) people's accuracy requirements for a production deployment.

-J
_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www.ietf.org/mailman/listinfo/bmwg


unces at ietf.org
Errors-To: bmwg-bounces at ietf.org

Bill Cerveny wrote:
>>> Wouldn't a big consideration be the quality and accuracy of the flow
>>> data?  How would the accuracy of the generated flow be evaluated?  The
>>
>>> flow generating capabilities of a network device is certainly less
>>> useful if the device's flow accuracy degrades when it is transiting
>>> more traffic.
>>
>> I would consider this out of the scope since to check that the testing
>> would
>> have to involve some external collector and the way those handle data
>> is even more complex than what happens on the network devices when
>> just registering the traffic.
>> I will think about it yet, but based on the testing we have done I don't
>> quite see how to do it.
>
> Couldn't you do something like send a known set of network traffic
> (defined by number of packets, types of packets, packet sizes, etc)
> across the router?  If the collector's interpretation of of the flow
> data matches what was sent across the router, a statement could be
> made about flow accuracy. If the collector's interpretation of the
> flow data does not match what was sent across the router, then a
> statement of accuracy cannot be made.  If the collector is unable to
> keep up with the flow data, perhaps another way of verifying the
> accuracy of the flow may be recording the flow data and playing it
> back at a rate that a collector could understand.
>
> I just think that making statements about a router's flow handling and
> reporting capabilities aren't complete if it can't be verified that
> the flow data reported is accurate.

This gets into the area of functionality verification and not
measurement, which seems to contradict the charter of the group (but I
could be misinterpreting).

Beyond this, there are many different variables involved, all of which
may have varying significance for different parties and can affect
accuracy in different ways.  It seems overly arduous to attempt to
identify a set (or multiple sets) of parameters that will satisfy all
(or even most) people's accuracy requirements for a production deployment.

-J
_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www.ietf.org/mailman/listinfo/bmwg