Internet Engineering Task Force Nevil Brownlee INTERNET-DRAFT The University of Auckland May 1998 Expires Nov 1998 SRL: A Simple Ruleset Language Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet-Drafts. This Internet Draft is a product of the Realtime Traffic Flow Measurement Working Group of the IETF. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a "working draft" or "work in progress." To view the entire list of current Internet-Drafts, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). Abstract This document describes a language for specifying rulesets, i.e. configuration files which may be loaded into a traffic flow meter so as to specify which traffic flows are measured by the meter, and the information it will store for each flow. Although the language is primarily intended for RTFM traffic flows, it may also be useful in other areas as a general way of specifying flows to be measured or collected. INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 Contents 1 Purpose and Scope 3 1.1 RTFM Meters and Traffic Flows . . . . . . . . . . . . . . . . 3 1.2 SRL Overview . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 SRL Language Description 4 2.1 Define Directive . . . . . . . . . . . . . . . . . . . . . . . 5 3 Statement 5 3.1 IF_statement . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.1 expression . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.2 factor . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.3 term . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.4 operand_list . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.5 operand . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.6 Test Part . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.7 Action Part . . . . . . . . . . . . . . . . . . . . . . . 8 3.2 Imperative_statement . . . . . . . . . . . . . . . . . . . . . 9 3.2.1 GOTO Statement . . . . . . . . . . . . . . . . . . . . . . 9 3.2.2 SAVE Statement . . . . . . . . . . . . . . . . . . . . . . 9 3.2.3 COUNT Statement . . . . . . . . . . . . . . . . . . . . . 10 3.2.4 IGNORE Statement . . . . . . . . . . . . . . . . . . . . . 10 3.2.5 NOMATCH Statement . . . . . . . . . . . . . . . . . . . . 10 3.2.6 STORE Statement . . . . . . . . . . . . . . . . . . . . . 11 3.2.7 RETURN Statement . . . . . . . . . . . . . . . . . . . . . 11 3.3 Subroutine_declaration . . . . . . . . . . . . . . . . . . . . 11 3.4 CALL_statement . . . . . . . . . . . . . . . . . . . . . . . . 12 4 Example Programs 13 4.1 Classify IP Port Numbers . . . . . . . . . . . . . . . . . . 13 4.2 Classify Traffic into Groups of Networks . . . . . . . . . . 14 5 APPENDICES 15 5.1 Appendix A: SRL Syntax in BNF . . . . . . . . . . . . . . . . 15 6 Acknowledgments 17 7 References 17 8 Author's Addresses 17 Nevil Brownlee [Page 2] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 1 Purpose and Scope A ruleset for an RTFM Meter is a sequence of instructions to be executed by the meter's Pattern Matching Engine (PME). The form of these instructions is described in detail in RFCs 2063 and 2064 [1], [2], but most users - at least inititially - find them confusing and difficult to write, mainly because the effect of each instruction is strongly dependent on the state of the meter's Packet Matching Engine at the moment of its execution. SRL is a procedural language for creating RTFM rulesets. It has been designed to be simple for people to understand, using statements which help to clarify the execution context in which they operate. SRL programs will be compiled into rulesets, which can then be downloaded to RTFM meters. 1.1 RTFM Meters and Traffic Flows The RTFM Architecture [1] defines a set of 'attributes' which apply to network traffic. Among the attributes are 'address attributes,' such as PeerType, PeerAddress, TransType and TransAddress, which have meaning for many protocols, e.g. for IP traffic (PeerType == IP) PeerAddress is an IP address, TransType is TCP, UDP, ICMP, etc., and TransAddress is usually an IP port number. An 'RTFM Traffic Flow' is simply a stream of packets observed by a meter as they pass across a network between two end points (or from a single end point). Each 'end point' of a flow is determined by the set of values of its address attributes. An 'RTFM Meter' is a measuring device - e.g. a program running on a Unix or PC host - which observes passing packets and builds 'Flow Data Records' for the flows of interest. RTFM traffic flows have another important property - they are bi-directional. This means that each flow data record in the meter has two sets of counters, one for packets travelling from source to destination, the other for returning packets. Within the RTFM architecture such counters appear as further attributes of the flow. An RTFM meter must be configured by the user, which means creating a 'Ruleset' so as to specify which flows are to be measured, and how much information (i.e. which attributes) should be stored for each of them. A rulset is effectively a program for a minimal virtual machine, the 'Packet Matching Engine (PME),' which is described in detail in [1]. An RTFM meter may run multiple rule sets, with every passing packet being Nevil Brownlee [Page 3] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 processed by each of the rulesets. The rule 'actions' in this document are described as though only a single ruleset were running. In the past creating a ruleset has meant writing machine code for the PME, which has proved rather difficult to do. SRL provides a high-level language which should enable users to create effective rulesets without having to understand the details of the PME. 1.2 SRL Overview An SRL program is executed from the beginning for each new packet arriving at the meter. It has two essential goals. (a) Decide whether the current packet is part of a flow which is of interest and, if necessary, determine its direction (i.e. decide which of its end-points is considered to be its source). Other packets will be ignored. (b) SAVE whatever information is required to identify the flow and accumulate (COUNT) quantitative information for that flow. At execution, the meter's Packet Matching Engine (PME) begins by using source and destination attributes as they appear 'on the wire.' If the attributes do not match those of a flow to be recorded, the PME will normally execute the program again, this time with the source and destination addresses interchanged. Because of this bi-directional matching, an RTFM meter is able to build up tables of flows with two sets of counters - one for forward packets, the other for backward packets. The programmer can, if required, suppress the reverse-direction matching and assign 'forward' and 'backward' directions which conform to the conventions of the external context. Goal (a) is achieved using IF statements which perform comparisons on information from the packet or from SRL variables. Goal (b) is achieved using one or more SAVE statements to store the flow's identification attributes; a COUNT statement then increments the statistical data accumulating for it. 2 SRL Language Description The SRL language is explained below using 'railway diagrams' to describe the syntax. Flow through a diagram is from left to right. The only exception to this is that lines carrying a left arrow may only be traversed right to left. In the diagrams, keywords are written in Nevil Brownlee [Page 4] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 capital letters; in practice an SRL compiler will be insensitive to case in keywords. Lower-case identifiers are explained in the text, or they refer to another diagram. The tokens of an SRL program obey the following rules: - Comments may appear on any line of an SRL program, following a # - White space is used to separate tokens - Semicolon is used as the statement terminator - Identifiers (e.g. for defines and labels) must start with a letter - Identifiers may contain letters, digits and underscores - The case of letters is not significant 2.1 Define Directive --- DEFINE -- defname ---- = ---- defined_text ------------------ ; Simple, parameterless, defines are supported, via the syntax above. The define name, defname, is an identifier made up of letters, digits and underscores. The defined text starts after the equal sign, and continues up to (but not including) the closing semicolon. (If a semicolon is required within define text, it must be preceded by a backslash). Wherever defname appears elsewhere in the program, it will be replaced by the defined text. For example, DEFINE telnet = 23; DEFINE smtp = 25; DEFINE http = (80, 8080); 3 Statement ----+-------------+-----+--- IF_statement -------------------+--- ; | | | | +-- label : --+ +--- Imperative_statement -----------+ | | | | +------<------+ +--- Subroutine_declaration ---------+ | | +--- CALL_statement -----------------+ Nevil Brownlee [Page 5] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 An SRL program is a sequence of SRL statements, each one terminated by a semicolon. There are four kinds of statements, as follows. Each statement may be labelled, i.e. preceded by a sequence of one or more labels. A label is an identifier, which must be followed by a semi-colon. Each statement is executed in sequence, unless one of them performs a GOTO, in which case execution transfers to the statement bearing the target label. Labels have a well-defined scope, within which they must be unique. Labels within a subroutine (i.e. between a SUBROUTINE and its matching ENDSUB) are local to that subroutine and are not visible outside it. Labels outside subroutines are part of a program's outer block. 3.1 IF_statement Test Part Action Part ............. ............... --- IF --------- expression -----------+-------- GOTO label -+--- ; | | +- SAVE , GOTO label -+ | | +- SAVE --------------+ | | +- IGNORE ------------+ | | +- NOMATCH -----------+ | | +- RETURN --+-------+-+ | | +-- n --+ 3.1.1 expression ------------+------------ factor -------------+-------------------- | | +-------------- || ---------------+ logical OR 3.1.2 factor ------------+------------- term --------------+-------------------- | | +-------------- && ---------------+ logical AND Nevil Brownlee [Page 6] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 3.1.3 term ------------+------- attrib == operand_list ---------+------------- | | +------------ ( expression ) ------------+ 3.1.4 operand_list ------------+-------------- operand -----------------+------------- | | +--- ( operand--+---------------+-- ) ---+ | | +-- , operand --+ | | +-------<-------+ 3.1.5 operand ------------- value ---------+----------------------+-------------- | | +------- / width ------+ | | +------- & mask -------+ 3.1.6 Test Part The IF statement evaluates a logical expression. If the expression value is TRUE, the action indicated by the keyword on the right of the diagram is executed. If the value is FALSE, the following statement is executed. The simplest form of expression is a test for equality (== operator); in this an RTFM attribute value (from the packet or from an SRL variable) is ANDed with a mask and compared with a value. More complicated expressions may be built up using parentheses and the && (logical AND) and || (logical OR) operators. Operand values may be specified as dotted decimal,hexadecimal or as a character constant (enclosed in apostrophes). Masks may be specified as numbers, dotted decimal e.g. &255.255.0.0 or hexadecimal e.g. &FF-FF-00-00 Nevil Brownlee [Page 7] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 or as a width in bits e.g. /16 If a mask is not specified, an all-ones mask is used. In SRL a value is always combined with a mask; this combination is referred to as an operand.For example, if we were interested in flows originating from IP network 130.216, we might write: IF SourcePeerAddress == 130.216.0.0 & 255.255.0.0 GOTO my_network; or equivalently IF SourcePeerAddress == 130.216/16 GOTO my_network; A list of values enclosed in parentheses may also be specified; the test succeeds if the masked attribute equals any of the values in the list. For example IF SourcePeerAddress == ( 130.216.7/24, 130.216.34/24 ) GOTO special_network; As this last example indicates, values are right-padded with zeroes, i.e. the given numbers specify the leading bytes of masks and values. The operand values and masks used in an IF statement must be consistent with the attribute being tested. For example, a four-byte value is acceptable as a peer address, but would not be accepted as a transport address (which may not be longer than two bytes). 3.1.7 Action Part A SAVE action saves attribute(s), mask(s) and value(s) as given in the statement. If the statement's expression tests more than one attribute, the masks and values are saved for all the attributes. For each value_list in the statement the value saved is the one which the packet actually matched. See below for further description of SAVE statements. Other actions are described in detail under "Imperative statements" below. Note that the RETURN action is valid only within subroutines. Nevil Brownlee [Page 8] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 3.2 Imperative_statement --+------------------------------------------- GOTO label ----+-- ; | | +-- SAVE attrib --+--+-----------+--+---+----------------+--+ | | | | | | | | | | +- / width -+ | +- , GOTO label -+ | | | | | | | | | +- & mask --+ | | | | | | | +--- = operand ---+ | | | +-- COUNT --------------------------------------------------+ | | +-- IGNORE -------------------------------------------------+ | | +-- NOMATCH ------------------------------------------------+ | | +-- RETURN --+-------+--------------------------------------+ | | | | | +-- n --+ | | | +-- STORE variable := value ------------+----------------+--+ | | +- , GOTO label -+ 3.2.1 GOTO Statement The GOTO statement (either on its own or as the last part of a larger statement) specifies the label of the statement to be executed next. 3.2.2 SAVE Statement The SAVE statement saves information which will (later) identify the flow in the meter's flow table. It does not actually record anything in the table; this is done when a subsequent COUNT statement executes. SAVE has two possible forms: SAVE attrib = operand saves the attribute, mask and value as given in the statement. This form of the SAVE statement is similar to that allowed in an IF statement, except that - since imperative statements do not perform a test - you may save an arbitrary value. Nevil Brownlee [Page 9] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 SAVE attrib SAVE attrib / width SAVE attrib & mask saves the attribute and mask from the statement, and the value resulting from their application to the current packet. This is most useful when used to save a value with a wider mask than than was used to select the packet. For example IF DestPeerAddress == 130.216/16 NOMATCH; IF SourcePeerAddress == 130.216/16 GOTO my_network; IGNORE; # Executes only if preceding # IF statements both fail. my_network: SAVE SourcePeerAddress /24; COUNT; 3.2.3 COUNT Statement The COUNT statement appears after all testing and saving is complete; it instructs the PME to build the flow identifier from the attributes which have been SAVEd, find it in the meter's flow table (creating a new entry if this is the first packet observed for the flow), and increment its counters. The meter then moves on to examine the next incoming packet. 3.2.4 IGNORE Statement The IGNORE statement terminates examination of the current packet without saving any information from it; the meter moves on to examine the next incoming packet, beginning again at the first statement of its program. 3.2.5 NOMATCH Statement The NOMATCH statement indicates that matching has failed for this execution of the program. If it is executed when a packet is being processed with its addresses in 'on the wire' order, the PME will perform the program again from the beginning with source and destination addresses interchanged. If it is executed following such an interchange, the packet will be IGNOREd. NOMATCH is illustrated in the above example, where it is used to ensure that flows having 130.216/16 as an end-point are counted as though 130.216 had been those flows' source peer (IP) address. Nevil Brownlee [Page 10] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 3.2.6 STORE Statement The STORE statement assigns a value to an SRL variable and SAVEs it. There are six SRL variables: SourceClass SourceKind DestClass DestKind FlowClass FlowKind Their names have no particular significance; they were arbitrarily chosen as likely RTFM attributes but can be used to store any integer values. Their values are set to zero each time examination of a new packet begins. 3.2.7 RETURN Statement The RETURN statement is used to return from subroutines and can be used only within the context of a subroutine. It is described in detail below (CALL statement). 3.3 Subroutine_declaration -- SUBROUTINE subname ( -+--+---ADDRESS ----pname---+--+- ) --> | | | | | +-- VARIABLE -- pname --+ | | | | | | +------<------- , ------+ | | | +-----------------------------+ >------+--- Imperative_statement ---+----- ENDSUB -------- ; | | +----IF_statement -----------+ | | +----CALL_statement ---------+ | | +-------------<--------------+ A Subroutine declaration has three parts: the subname is an indentifier, used to name the subroutine. Nevil Brownlee [Page 11] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 the Parameter list specifies the subroutine's parameters. Each parameter is preceded with a keyword indicating its type - VARIABLE indicates an SRL variable (see the STORE statement above), ADDRESS indicates any other RTFM attribute. The parameter name (pname in the diagram) must be the name of a meter 'parameter' variable, i.e. P1, P2, P3, P4 or P5. The meter implements these as global variables, which means that the SRL programmer must be careful to avoid conflicts when calling one subroutine from another. the Body specifies what processing the subroutine will perform. This is simply a sequence of Imperative, IF and CALL statements, terminated by the ENDSUB keyword. Note that GOTOs in a subroutine may not refer to labels outside it. The only way to leave a subroutine is via a RETURN statement. 3.4 CALL_statement --- CALL subname ( -+--+-- parameter --+--+- ) --> | | | | | +---<---- , ----+ | | | +---------------------+ >---+--- n: Imperative_statement ---+---- ENDCALL -------- ; | | +---------------<---------------+ The CALL statement invokes an SRL subroutine. The parameters are SRL variables or other RTFM attributes, and their types must match those in the subroutine declaration. Following the parameters is a sequence of statements, each preceded by an integer label. These labels will normally be 1:, 2:, 3:, etc, but they do not have to be contiguous. They are referred to in RETURN statements. e.g. RETURN 2; would return to the statement labelled 2: in the subroutine call. If this statement does not execute a GOTO, execution will then continue with the first statement after ENDCALL. Nevil Brownlee [Page 12] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 If the return statement does not specify a return label, the first statement executed after RETURN will be the statement immediately following ENDCALL. 4 Example Programs 4.1 Classify IP Port Numbers # SRL program to classify IP port numbers # IF SourcePeerType == IP SAVE, GOTO IP_pkt; IGNORE; # Not an IP packet # IP_pkt: IF SourceTransType == ( tcp, udp ) SAVE, GOTO tcp_udp; GOTO fin; # Not tcp or udp (probably doesn't have ports) # tcp_udp: IF SourceTransAddress == ( www, ftp, telnet ) NOMATCH; # IF DestTransAddress == www GOTO c_www; IF DestTransAddress == ftp GOTO c_ftp; IF DestTransAddress == telnet GOTO c_telnet; # GOTO fin; # Count as 'unknown' # c_www: STORE FlowKind := 'W', GOTO fin; c_ftp: STORE FlowKind := 'F', GOTO fin; c_telnet: STORE FlowKind := 'T', GOTO fin; # fin: SAVE SourcePeerAddress /32; SAVE DestPeerAddress /32; COUNT; This program counts only IP packets, saving SourceTransType (tcp, udp or 0), Source- and DestPeerAddress (32-bit IP addresses) and FlowKind ('W' for www, 'F' for ftp, 'T' for telnet, 0 for inclassified). The program uses NOMATCH actions to specify the packet direction - its resulting flows will have the well-known ports as their destination. Nevil Brownlee [Page 13] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 4.2 Classify Traffic into Groups of Networks # SRL program to classify traffic into network groups # CALL net_kind (SourcePeerAddress, SourceKind) ENDCALL; CALL net_kind (DestPeerAddress, DestKind) ENDCALL; COUNT; # SUBROUTINE net_kind (ADDRESS p1, VARIABLE p2) IF p1 == 130.216/16 SAVE, GOTO nk_mysite; IF p1 == ( 130.217/16, 130.123/16, 130.195/16, 132.181/16, 138.75/16, 139.80/16 ) SAVE, GOTO nk_mynetwork; SAVE p1 /24; # Not my site or my network STORE p2 := 30; RETURN 3; nk_mysite: STORE p2 := 10; RETURN 1; nk_mynetwork: STORE p2 := 20; RETURN 2; ENDSUB; The net_kind subroutine determines whether p1 is my network (130.216), one of the networks in my network (one of the networks in the list), or some other network. It saves the network address from p1 (16 bits for my site and my network, 24 bits for others), stores a value of 10, 20 or 30 in p2, and returns to 1:, 2: or 3:. net_kind is called twice, saving Source- and DestPeerAddress and Source- and DestKind; the COUNT statement produces flows identified by these four RTFM attributes, with no particular source-dest ordering. In the program no use is made of return numbers amd they could have been omitted. However, we might wish to re-use the subroutine in another program doing different things for different return numbers, as in the version below. CALL net_kind (DestPeerAddress, DestKind) 1: NOMATCH; ENDCALL; CALL net_kind (SourcePeerAddress, SourceKind) 1: COUNT; # site -> network or other ENDCALL; SAVE SourcePeerAddress /24; SAVE DestPeerAddress /24; COUNT; Nevil Brownlee [Page 14] INTERNET-DRAFT SRL: A Simple Ruleset Language May 1998 This version uses a NOMATCH statement to ensure that site -> network or other flows have site as their source. The NOMATCH also rejects site -> site traffic. Traffic which doesn't have site as source or destination saves 24 bits of its addresses (the subroutine might only have saved 16) before counting such an unusual flow. 5 APPENDICES 5.1 Appendix A: SRL Syntax in BNF ::= | ::=