David, 

You make a good point on the end use of syslog messages. I'm not a big
fan of large messages, I'd be happy limiting it to 64K even over TCP. It
makes it readable by humans and parsable by standard file parsing
programs.

Some standard text editors will only allow a max line length of 4096
characters.

If we are to assume that a "syslog" message is intended to be displayed
on a console, logged to a file and maybe included in some sort of
paging/e-mail alert system, then the message size really needs to be
kept fairly short.

Rainer's experience is that some devices/apps will send very large
diagnostic messages of around 1MB in size.

I can understand the need in this case to be able to handle large
message sizes. However, I don't plan on accepting messages any larger
than 64K in my implementation of -protocol.

A client would have to make a REALLY good case for me to want to accept
larger messages, especially 16MB in size. FTP, or even HTTP POST would
be the best solution in this case. 

Syslog messages are meant for displaying, logging to disk and alerting.
Not large binary style core dump info or database style records.

I would like to see a note in the -protocol spec that encourages
implementers to only send small messages that adhere to the original
spirit of syslog.

Cheers

Andrew




-----Original Message-----
From: syslog-sec-bounces@www.employees.org
[mailto:syslog-sec-bounces@www.employees.org] On Behalf Of David B
Harrington
Sent: Thursday, 14 October 2004 4:10 a.m.
To: 'Rainer Gerhards'; 'Anton Okmianski'; syslog-sec@employees.org
Subject: [Syslog-sec] Maximum message size 


Hi,

I am concerned that we are in danger of promoting uses of syslog that
will defeat its current usefulness as a human-readable log of system
activity.

Do network operators really believe the 16,777,216 octet size is a
needed improvement to syslog, or are we designing this size in
explicitly for the vendors of applications who want to use syslog as a
programmatic interface? I believe Syslog should be designed to meet
the needs of operators. Most of the discussion ssems to be from
vendors of syslog receivers or applications. Other protocols such as
FTP might be better used for such a specialty use case. 

Does allowing messages of 16,777,216 octets to be accumulated within
the system log make it harder to use some widely-available minimal
text editors, like Notepad, to view the system log? Will having huge
application-specific messages in the system log make it harder for an
operator to locate useful troubleshooting messages within the system
log? Can the information in such a huge message fit within an 80x25
screen, when an operator is troubleshooting network problems and needs
to use a bare-bones terminal attached to the serial port of a device
to view the system log? 

Syslog was designing to be an operator's tool, and Syslog should be
designed to meet the needs of operators.  Will allowing messages of
this size negatively impact its usefulness in the primary use case?  

What do network **operators** think about the need for messages of
this size? Have we deliberately asked them their opinion on this by,
for example, taking a survey or going to NANOG to ask them?

I don't think we should support such large messages in the system
logging protocol.


David Harrington
dbharrington@comcast.net



-----Original Message-----
From: syslog-sec-bounces@www.employees.org
[mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer
Gerhards
Sent: Wednesday, October 13, 2004 5:17 AM
To: Anton Okmianski; syslog-sec@employees.org
Subject: RE: [Syslog-sec] UDP message size issue proposal

Anton,

I agree to your conclusion.

Although I am the one who always voted for larger sizes, the
discussion has shown that this yields to unnecessary complexity. I can
satisfy my needs with a vendor-extension structured data element,
which I think shows the flexibility of the new drafts.

So I support your move. I would tend to even remove the transport
version header. If there is need to, we could always include that in a
later release (e.g. "v1", which would make it clearly distingshable
from the current frame format). I see little need to do so, though.

Regarding -protocol, I think we should still keep an upper limit in
the spec. Even I don't see any reason why a syslog message should go
above 16MB. So for -protocol, I propose the following new text:

####
5.1  Message Length

   The maximum length of any syslog message is 16,777,216 octets.  Any
   receiver receiving a larger message MUST discard the message.  A
   diagnostic message SHOULD be logged in this case.

   A receiver MUST be able to accept messages that are 480 octets (or
   less) in length.  A receiver SHOULD be able to accept messages that
   are 65,535 octets (or less) in length.  It is RECOMMENDED that
   receivers be able to accept messages up to the maximum message
length
   of 16,777,216 octets.

   If a receiver receives a message within the maximum length, but
with
   a length larger than it handles, the receiver MAY discard or
truncate
   it.

   A transport mapping MAY define a maximum length below the one
   specified here.  Each transport mapping MUST support a maximum size
   of 480 octets or more.
####

If there is agreement on this, I can post a new version of -protocol.
That version will also include some changes made thanks to Andrew Ross
comments (sent via private mail). I have finished this version. It is
available for immediate posting, but I would like to wait for
agreement on the size issue (at least if that can be reached quickly).

Rainer

> -----Original Message-----
> From: syslog-sec-bounces@www.employees.org
> [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton 
> Okmianski
> Sent: Tuesday, October 12, 2004 11:25 PM
> To: syslog-sec@employees.org
> Subject: [Syslog-sec] UDP message size issue proposal
> 
> Hi!
> 
> Sorry for a long delay on this issue - I was on a 2 week vacation.
I 
> have spoken with a number of TCP/UDP/IP experts regarding the sizing

> issue.  I am ready to propose the following changes:
> 
> 1. Syslog-protocol will state that the max message will be defined
by 
> the transport layer.
> 
> 2. Syslog-transport-udp will support messages up to maximum UDP 
> datagram size of 64K. UDP is a very bad choice for large message 
> transmissions, so it does not make sense for us to stretch it by 
> adding our own fragmentation without other transmission control 
> features such as found in TCP.
> 
> 3. Syslog-transport-udp will rely on IP fragmentation and we will
get 
> rid of "proprietary" fragmentation which was designed to handle 
> messages over 64K and deal with various
> non-compliant network hosts.    
> 
> 4. Syslog-transport-udp will recommend sending messages within the 
> boundaries of prevalent MTU size on a given network to be safe. It 
> will recommend Ethernet's 1500 bytes less headers and will draw
reader 
> attention to the minimum MTU size hosts on the network are required
to 
> support for
> IPv6 (576 bytes) and IPv6 (1280 bytes).   
> 
> 5. Path MTU discovery may not work robustly and some TCP/IP stacks
may 
> not support UDP packets of full 64K size and truncate them.  We will

> mention this and bite this bullet.
> We should not restrict the protocol due to incompliant
implementations 
> because it is a moving target and penalizes compliant
implementations 
> with extra overhead. The above size recommendation would partially 
> deal with this, but leave the
> final choice up to the administrator.      
> 
> 6. We will get rid of most syslog transport headers for UDP as they 
> will no longer be needed.  The only thing that will be left is the 
> transport protocol version prefixed to every syslog message.  Should

> we even bother with that?
> 
> This is a major change to the syslog-transport-udp.  I'd like to get

> positive feedback before I proceed with this update.
> The best part is that if we all agree on the above, the next draft 
> will be 1/3 of the size -- easier read for you. :)
> 
> Thanks,
> Anton. 
>     
> 
> _______________________________________________
> Syslog-sec mailing list
> Syslog-sec@www.employees.org
> http://www.employees.org/mailman/listinfo/syslog-sec
> 
_______________________________________________
Syslog-sec mailing list
Syslog-sec@www.employees.org
http://www.employees.org/mailman/listinfo/syslog-sec


_______________________________________________
Syslog-sec mailing list
Syslog-sec@www.employees.org
http://www.employees.org/mailman/listinfo/syslog-sec

_______________________________________________
Syslog-sec mailing list
Syslog-sec@www.employees.org
http://www.employees.org/mailman/listinfo/syslog-sec