Re: [Sipping] Alternate CLF syntax proposal
Theo Zourzouvillys <theo@crazygreek.co.uk> Sun, 29 March 2009 10:25 UTC
Return-Path: <theo@crazygreek.co.uk>
X-Original-To: sipping@core3.amsl.com
Delivered-To: sipping@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 85D7F3A6ADA for <sipping@core3.amsl.com>; Sun, 29 Mar 2009 03:25:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.519
X-Spam-Level:
X-Spam-Status: No, score=-5.519 tagged_above=-999 required=5 tests=[AWL=-0.142, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_24=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id veoQDYuTwA0j for <sipping@core3.amsl.com>; Sun, 29 Mar 2009 03:25:58 -0700 (PDT)
Received: from exprod7og127.obsmtp.com (exprod7og127.obsmtp.com [64.18.2.210]) by core3.amsl.com (Postfix) with SMTP id 2A3053A69B4 for <sipping@ietf.org>; Sun, 29 Mar 2009 03:25:58 -0700 (PDT)
Received: from source ([72.14.220.155]) by exprod7ob127.postini.com ([64.18.6.12]) with SMTP ID DSNKSc9M7qiPh/PrjtvFNorUPckWq44403G2@postini.com; Sun, 29 Mar 2009 03:26:55 PDT
Received: by fg-out-1718.google.com with SMTP id 22so182955fge.14 for <sipping@ietf.org>; Sun, 29 Mar 2009 03:26:54 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <c164605b0903260836p45a8bd8eg9ad50f670ef82302@mail.gmail.com>
References: <49CAE21C.5060309@nostrum.com> <c164605b0903260836p45a8bd8eg9ad50f670ef82302@mail.gmail.com>
Date: Sun, 29 Mar 2009 11:26:39 +0100
Received: by 10.86.68.1 with SMTP id q1mr1063797fga.62.1238322414233; Sun, 29 Mar 2009 03:26:54 -0700 (PDT)
Message-ID: <167dfb9b0903290326t519e6df8naacdb396a259f8f2@mail.gmail.com>
From: Theo Zourzouvillys <theo@crazygreek.co.uk>
To: Jason Fischl <jason.fischl@gmail.com>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Cc: sipping WG <sipping@ietf.org>, Adam Roach <adam@nostrum.com>, draft-gurbani-sipping-clf@tools.ietf.org
Subject: Re: [Sipping] Alternate CLF syntax proposal
X-BeenThere: sipping@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: "SIPPING Working Group \(applications of SIP\)" <sipping.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/sipping>, <mailto:sipping-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sipping>
List-Post: <mailto:sipping@ietf.org>
List-Help: <mailto:sipping-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sipping>, <mailto:sipping-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 29 Mar 2009 10:25:59 -0000
On Thu, Mar 26, 2009 at 4:36 PM, Jason Fischl <jason.fischl@gmail.com> wrote: > I quite like this scheme. It is very simple to generate and parse and will > be blazingly fast. It would also be very simple to create a utility to > generate "human-readable" versions from it which would address that > particular concern. I also really like Adam's binary proposal and would much prefer we progress down a binary path instead of plain text. My reasoning is that the point of standardising such a format would be for analysis or monitoring/diagnostics, which would inherently be processed by machines, not humans. The only advantage of a text format is for tail -f'ing on a live server; such a thing has no reason to be standardised short of the operator in question knowing which fields are which in the same way i do when tailing an access.log. This is commonly done by something such as: LogFormat "%h %l %u %t \"%r\" %>s %b" common and can easily be changed to include other stuff on a per-server/implementation basis, i.e: LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" VLOG=%{VLOG}e" vhost However, the format which would be processed by collectors, stored, analysed, and possibly post-processed or merged with others would really want to be binary. We currently use our own format internally for transmitting between exporters and collectors and it would be nice to see a standardised format so we can perhaps build some open source analysis tools that would work with multiple vendors or a collector that provides nice network statistics. > Comments about the format: > - add a version number to the header > - set aside a range of tags for vendor attributes. yup, and in addition, i would like to see an exporter identifier. > A few miscellaneous comments about contents: > - for sent messages: local ip:port used to send the message + transport used > + destination ip:port > - for received messages: remote ip:port message was received from + local > ip:port + transport used > - it may be a good idea to have a predefined tag which can be used as an > index into another diagnostic file. e.g. pointer into a PCAP file or vendor > diagnostic log. > - Is there anything we want to jam in here to help debug TCP/TLS issues or > is that out of scope? .. infact, come to think of it, IPFIX (RFC5101) seems perfect for all of this - It would even allow encoding lower level data should you want to such as ICMP type/code, source/dst IP, interfaces, or TCP/TLS packet, SYN/ACK/RST events etc. all we'd need to do is define a bunch of IEs in a document, and the rest would "just work" (tm). ~ Theo Sent from: Bicester Oxfordshire United Kingdom.
- [Sipping] Alternate CLF syntax proposal Adam Roach
- Re: [Sipping] Alternate CLF syntax proposal Vijay K. Gurbani
- Re: [Sipping] Alternate CLF syntax proposal Jason Fischl
- Re: [Sipping] Alternate CLF syntax proposal Hadriel Kaplan
- Re: [Sipping] Alternate CLF syntax proposal Hadriel Kaplan
- Re: [Sipping] Alternate CLF syntax proposal Hadriel Kaplan
- Re: [Sipping] Alternate CLF syntax proposal Theo Zourzouvillys
- Re: [Sipping] Alternate CLF syntax proposal Jiri Kuthan
- Re: [Sipping] Alternate CLF syntax proposal Theo Zourzouvillys
- Re: [Sipping] Alternate CLF syntax proposal Cullen Jennings
- Re: [Sipping] Alternate CLF syntax proposal Daryl Malas
- Re: [Sipping] Alternate CLF syntax proposal Hadriel Kaplan
- Re: [Sipping] Alternate CLF syntax proposal Hadriel Kaplan
- Re: [Sipping] Alternate CLF syntax proposal Theo Zourzouvillys
- Re: [Sipping] Alternate CLF syntax proposal Hisham Khartabil
- Re: [Sipping] Alternate CLF syntax proposal Vijay Gurbani
- Re: [Sipping] Alternate CLF syntax proposal Dale Worley
- Re: [Sipping] Alternate CLF syntax proposal Adam Roach
- Re: [Sipping] Alternate CLF syntax proposal Adam Roach