idnits 2.17.1 draft-ietf-forces-netlink-01.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 28 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 28 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 22 instances of too long lines in the document, the longest one being 11 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 107 has weird spacing: '...Netlink layer...' == Line 358 has weird spacing: '...Netlink layer...' == Line 609 has weird spacing: '...address inter...' == Line 626 has weird spacing: '...sioning servi...' == Line 638 has weird spacing: '...ngth of the ...' == (14 more instances...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 2001) is 8192 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC-2119' is mentioned on line 40, but not defined == Unused Reference: 'RFC1633' is defined on line 1032, but no explicit reference was found in the text == Unused Reference: 'RFC1812' is defined on line 1039, but no explicit reference was found in the text == Unused Reference: 'RFC2475' is defined on line 1042, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 1633 ** Downref: Normative reference to an Informational RFC: RFC 2475 ** Downref: Normative reference to an Historic RFC: RFC 1157 ** Obsolete normative reference: RFC 3036 (Obsoleted by RFC 5036) Summary: 13 errors (**), 0 flaws (~~), 13 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ForCES Working Group Jamal Hadi Salim 3 Internet Draft Znyx Networks 4 Hormuzd Khosravi 5 Intel 6 Andi Kleen 7 Suse 8 Alexey Kuznetsov 9 INR/Swsoft 10 November 2001 12 Netlink as an IP services protocol 14 draft-ietf-forces-netlink-01.txt 16 Status of this Memo 18 This document is an Internet-Draft and is in full conformance with 19 all provisions of Section 10 of RFC2026. Internet-Drafts are working 20 documents of the Internet Engineering Task Force (IETF), its areas, 21 and its working groups. Note that other groups may also distribute 22 working documents as Internet-Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as ``work in progress.'' 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 Conventions used in this document 37 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 38 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 39 this document are to be interpreted as described in [RFC-2119]. 41 1. Abstract 43 This document describes Linux Netlink, which is used in Linux both 44 as an inter-kernel messaging system as well as between kernel and 46 jhs_hk_ak_ank draft-forces-netlink-01.txt 48 user-space. The purpose of this document is intended as informa- 49 tional in the context of prior art for the ForCES IETF working 50 group. The focus of this document is to describe netlink from a 51 context of a protocol between a Forwording Engine Component (FEC) 52 and a Control Plane Component(CPC) that define an IP service. 54 The document ignores the ability of netlink as a inter-kernel mes- 55 saging system, as a an inter-process communication scheme (IPC) or 56 its use in configuring other non-network as well as network but 57 non-IP services (such as decnet etc). 59 2. Introduction 61 The concept of IP Service control-forwarding separation was first 62 introduced in the early 1980s by the BSD 4.4 routing sock- 63 ets[stevens]. The focus at that time was a simple IP(v4) forward- 64 ing service and how the CPC, either via a command line configura- 65 tion tool or a dynamic route daemon, can control forwarding tables 66 for that IPV4 forwarding service. 68 The IP world has evolved considerably since those days. Linux 69 netlink, when observed from a service provisioning point of view 70 takes routing sockets one step further by breaking the barrier of 71 focus around IPV4 forwarding. Since the linux 2.1 kernel, netlink 72 has been providing the IP service abstraction to a few services 73 other than the classical IPv4 forwarding. 75 We first give some concept definitions and then describe how 76 netlink fits in. 78 2.1. Some definitions 80 A Control plane(CP) is an execution environment that may have sev- 81 eral components which we refer to as CPCs. Each CPC provides con- 82 trol for a different IP service being executed by a FE component. 83 This means that there might be several CPCs on a physical CP if it 84 is controlling several IP services. In essence, the cohesion 85 between a CP component and a FE component is the service abstrac- 86 tion. 88 In the diagram below we show a simple FE<->CP setup to provide an 89 example of the classical IPv4 service with an extension to do some 90 basic QoS egress scheduling and how it fits in this described 92 jhs_hk_ak_ank draft-forces-netlink-01.txt 94 model. 96 Control Plane (CP) 97 .------------------------------------ 98 | /^^^^^ /^^^^^ | 99 | | | | COPS |- | 100 | | ospfd | | PEP | | 101 | / _____/ | | 102 /------_____/ | | | 103 | | | | | | 104 | |_____________________|___|_________| 105 | | | | 106 ****************************************** 107 Forwarding ************* Netlink layer ************ 108 Engine (FE) ***************************************** 109 .-------------|-----------|------------|---|----------- 110 | IPv4 forwading | / | 111 | FE Service / / | 112 | Component / / | 113 | ---------------/---------------/--------- | 114 | | | / | | 115 packet | | --------|-- ----|----- | packet 116 in | | | IPV4 | | Egress | | out 117 -->--->|------>|---->|Forwading |----->| QoS |--->| ---->|----> 118 | | | | | Scheduler| | | 119 | | ----------- ---------- | | 120 | | | | 121 | --------------------------------------- | 122 | | 123 ------------------------------------------------------- 125 2.1.1. Control Plane Components (CPCs) 127 Control plane components would encompass signalling protocols with 128 diversity ranging from dynamic routing protocols such as OSPF 129 [RFC2328] to tag distribution protocols such as CR-LDP [RFC3036]. 130 Classical Management protocols and activities also fall under this 131 category. These include SNMP [RFC1157], COPS [RFC2748] or propri- 132 etary CLI/GUI configuration mechanisms. 134 jhs_hk_ak_ank draft-forces-netlink-01.txt 136 The purpose of the control plane is to provide an execution envi- 137 ronment for the above mentioned activities with the ultimate goal 138 being to configure and manage the second NE component: the FE. The 139 result of the configuration would define the way packets travesing 140 the FE are treated. 142 In the above diagram, ospfd and COPS are distinct CPCs. 144 2.1.2. Forwarding Engine Components 146 The FE is the entity of the NE that incoming packets (from the net- 147 work into the NE) first encounter. 149 The FE's service specific component massages the packet to provide 150 it with a treatment to achieve a IP service as defined by the con- 151 trol plane components for that IP service. Different services will 152 utilize different FEC. Service modules maybe chained to achieve a 153 more complex service (as shown in the diagram). When built for 154 providing a specific service, the FE service component will adhere 155 to a Forwading Model. 157 In the above diagram, the IPV4 FE component includes both the IPV4 158 Forwarding service module as well as the Egress Scheduling service 159 module. Another service might may add a policy forwarder between 160 the IPV4 forwader and the QoS egress Scheduler. A simpler classi- 161 cal service would have constituted only the IPV4 forwarder. 163 2.1.3. IP Services 165 An IP Service is the treatment of an IP packet within the NE. This 166 treatment is provided by a combination of both the CPC and FEC 168 The time span of the service is from the moment when the packet 169 arrives at the NE to the moment it departs. In essence an IP ser- 170 vice in this context is a Per-Hop Behavior. A service control/sig- 171 naling protocol/management-application (CP components running on 172 NEs defining the end to end path) unifies the end to end view of 173 the IP service. As noted above, these CP components then define the 174 behavior of the FE (and therefore the NE) to a described packet. 176 A simple example of an IP service is the classical IPv4 Forwarding. 177 In this case, control components such as routing protocols(OSPF, 179 jhs_hk_ak_ank draft-forces-netlink-01.txt 181 RIP etc) and proprietary CLI/GUI configurations modify the FE's 182 forwarding tables in order to offer the simple service of forward- 183 ing packets to the next hop. Traditionally, NEs offering this sim- 184 ple service are known as routers. 186 Over the years it has become important to add aditional services to 187 the routers to meet emerging requirements. More complex services 188 extending classical forwarding were added and standardized. These 189 newer services might go beyond the layer 3 contents of the packet 190 header. However, the name "router", although a misnomer, is still 191 used to describe these NEs. Services (which may look beyond the 192 classical L3 headers) here include firewalling, Qos in Diffserv and 193 RSVP, NATs, policy based routing etc. Newer control protocols or 194 management activities are introduced with these new services. 196 One extreme definition of a IP service is something a service 197 provider would be able to charge for. 199 3. Netlink Architecture 201 IP services components control is defined by using templates. 203 The FEC and CPC participate to deliver the IP service by communi- 204 cating using these templates. The FEC might continously get 205 updates from the control plane component on how to operate the ser- 206 vice (example for V4 forwarding route additions or deletions). 208 The interaction between the FEC and the CPC, in the netlink con- 209 text, would define a protocol. Netlink provides the mechanism for 210 the CPC(residing in user space) and FEC(residing in kernel space) 211 to define their own protocol definition. Kernel space and user 212 space just mean different protection domains direct where direct 213 memory access is not allowed inbetween. Therefore a wire protocol 214 is needed to communicate. The wire protocol would be normally be 215 provided by some privileged service that is able to copy between 216 multiple protection domains. We will call this service netlink 217 service. Netlink service could also be mapped to a different 218 transport layer if the CPC should be running on a different node 219 than the CPC. The FEC and CPC, using netlink mechanisms, may 220 choose to define a reliable protocol between each other, for exam- 221 ple. By default netlink provides an unreliable communication. 223 Note that the FEC and CPC can both live in the same memory protec- 224 tion domain and use the connect() system call to create a path to 225 the peer and talk to each other. We will not discuss this further 227 jhs_hk_ak_ank draft-forces-netlink-01.txt 229 other than to say it is available as a mechanism. Through out this 230 document we will refer interchangbly to the FEC to mean kernel- 231 space and the CPC to mean user-space. 233 Note: Netlink allows participation in IP services by both service 234 components. 236 3.1. Netlink Logical model 238 In the diagram below we show a simple FEC<->CPC logical relation- 239 ship. We use the example of IPV4 forwarding FEC (NETLINK_ROUTE, 240 which is discussed further below) as an example. 242 Control Plane (CP) 243 .------------------------------------ 244 | /^^^^^ /CPC-2 | 245 | | CPC-1 | | COPS | | 246 | | ospfd | | PEP | | 247 | / _____/ | 248 | _____/ | | 249 | | | | 250 ****************************************| 251 ************* BROADCAST WIRE ************ 252 FE---------- *****************************************. 253 | IPv4 forwading | | / | 254 | FEC | | | | 255 | --------------/-----|-----------|-------- | 256 | | / | | | | 257 | | .-------. .-------. .------. | | 258 | | |ingress| | IPV4 | |Egress| | | 259 | | |police | |Forward| | QoS | | | 260 | | |_______| |_______| |Sched | | | 261 | | ------ | | 262 | --------------------------------------- | 263 | | 264 ----------------------------------------------------- 266 Netlink logically models FECs and CPCs in the form of nodes inter- 267 connected to each other via a broadcast wire. 269 The wire is specific to a service. The example above shows the 270 broadcast wire belonging to the extended IPV4 forwarding service. 272 jhs_hk_ak_ank draft-forces-netlink-01.txt 274 Nodes connect to the wire and register to receive specific mes- 275 sages. CPCs may connect to multiple wires if it helps them to con- 276 trol the service better. All nodes(CPCs and FECs) dump packets on 277 the broadcast wire. Packets could be discarded by the wire if mal- 278 formed or not specifically formated for the wire. Dropped packets 279 are not seen by any of the nodes. The netlink service MAY signal 280 an error to the original if it detects an malformatted netlink 281 packet. 283 Packets sent on the wire could be broadcast, multicast or unicast. 284 FECs or CPCs pick specific messages of interest for processing or 285 just monitoring purposes. 287 3.2. The message format 289 There are three levels to a netlink message: The general netlink 290 message header, the IP service specific template, the IP service 291 specific data. 293 0 1 2 3 294 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 296 | | 297 | Netlink message header | 298 | | 299 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 | | 301 | IP Service Template | 302 | | 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | | 305 | IP Service specific data in TLVs | 306 | | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 309 3.3. Protocol Model 311 This section expands on how netlink provides the mechanism for ser- 312 vice oriented FEC and CPC interaction. 314 jhs_hk_ak_ank draft-forces-netlink-01.txt 316 3.3.1. Service Addressing 318 Access is provided by first connecting to the service on the FE. 319 This is done by making a socket() system call to the PF_NETLINK 320 domain. Each FEC is identified by a protocol number. One may open 321 either SOCK_RAW or SOCK_DGRAM type sockets although netlink doesnt 322 distinguish the two. The socket connection provides the basis for 323 the FE<->CP addressing. 325 Connecting to a service is followed (at any point during the life 326 of the connection) by issuing either a service specific command 327 mostly for configuration purposes (from the CPC to the FEC) or sub- 328 scribing/unsubscribing to service(s') events. 330 3.3.1.1. Sample Service Hierachy 332 In the diagram below we show a simple IP service, foo, and the 333 interaction it has between CP and FE components for the ser- 334 vice(labels 1-3). 336 We introduce the diagram below to demonstrate CP<->FE addressing. 337 In this section we illustrate only the addressing semantics. In 338 section 4, the diagram is referenced again to define the protocol 339 interaction between srevice foo's CPC and FEC (labels 4-10). 341 jhs_hk_ak_ank draft-forces-netlink-01.txt 343 CP 344 [--------------------------------------------------------. 345 | .-----. | 346 | | . -------. | 347 | | CLI | / | 348 | | | | CP protocol | 349 | /->> -. | component | <-. | 350 | __ _/ | | For | | | 351 | | | IP service | ^ | 352 | Y | foo | | | 353 | | ___________/ ^ | 354 | Y 1,4,6,8,9 / ^ 2,5,10 | 3,7 | 355 --------------- Y------------/---|----------|----------- 356 | ^ | ^ 357 **|***********|****|**********|********** 358 ************* Netlink layer ************ 359 **|***********|****|**********|********** 360 FE | | ^ ^ 361 .-------- Y-----------Y----|--------- |----. 362 | | / | 363 | Y / | 364 | . --------^-------. / | 365 | |FE component/module|/ | 366 | | for IP Service | | 367 --->---|------>---| foo |----->-----|------>-- 368 | ------------------- | 369 | | 370 | | 371 ------------------------------------------ 373 The control plane protocol for IP service foo does the following to 374 connect to its FE counterpart. The steps below are also numbered 375 above in the diagram. 377 1) Connect to IP service foo through a socket connect. A typical con- 378 nection would be via a call to: socket(AF_NETLINK, SOCK_RAW, 379 NETLINK_FOO) 381 2) Bind to listen to specific async events for service foo 383 3) Bind to listen to specific async FE events 385 jhs_hk_ak_ank draft-forces-netlink-01.txt 387 3.3.2. Netlink message header 389 Netlink messages consist of a byte stream with one or multiple 390 Netlink headers and associated payload. If the payload is too big 391 to fit into a single message it can be split over multiple netlink 392 messages. This is called a multipart message. For multipart mes- 393 sages the first and all following headers have the NLM_F_MULTI 394 netlink header 395 flag set, except for the last header which has the netlink header 396 type NLMSG_DONE. 398 The netlink message header is shown below. 400 0 1 2 3 401 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 402 0 1 2 3 403 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 404 | Length | 405 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 406 | Type | Flags | 407 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 408 | Sequence Number | 409 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 410 | Process PID | 411 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 413 The fields in the header are: 415 jhs_hk_ak_ank draft-forces-netlink-01.txt 417 Length: 32 bits 418 The length of the message in bytes including the header. 420 Type: 16 bits 421 This field describes the message content. 422 It can be one of the standard message types: 423 NLMSG_NOOP message is ignored 424 NLMSG_ERROR the message signals an error and the payload 425 contains a nlmsgerr structure. This can be looked 426 at as a NACK and typically it is from FEC to CPC. 427 NLMSG_DONE message terminates a multipart message 429 Individual IP Services specify more message types, for e.g., 430 NETLINK_ROUTE Service specifies several types such as RTM_NEWLINK, 431 RTM_DELLINK, RTM_GETLINK, RTM_NEWADDR, RTM_DELADDR, RTM_NEWROUTE, 432 RTM_DELROUTE, etc. 434 Flags: 16 bits 435 The standard flag bits used in netlink are 436 NLM_F_REQUEST Must be set on all request messages (typically 437 from user space to kernel space) 438 NLM_F_MULTI Indicates the message is part of a multipart 439 message terminated by NLMSG_DONE 440 NLM_F_ACK Request for an acknowledgment on success. 441 Typical direction of request is from user 442 space to kernel space. 443 NLM_F_ECHO Echo this request. Typical direction of 444 request is from user space to kernel space. 446 Additional flag bits for GET requests on config information in 447 the FEC. 448 NLM_F_ROOT Return the complete table instead of a 449 single entry. 450 NLM_F_MATCH Return all matching criteria passed in 451 message content 452 NLM_F_ATOMIC Return an atomic snapshot of the table being 453 referenced. This may require special privileges 454 because it has the potential to interrupt 455 service in the FE for a longer time. 457 Convenience macros for flag bits: 458 NLM_F_DUMP This is NLM_F_ROOT or'ed with NLM_F_MATCH 460 Additional flag bits for NEW requests 461 NLM_F_REPLACE Replace existing matching config object with 462 this request. 463 NLM_F_EXCL Don't replace the config object if it already 465 jhs_hk_ak_ank draft-forces-netlink-01.txt 467 exists. 468 NLM_F_CREATE Create config object if it doesn't already 469 exist. 470 NLM_F_APPEND Add to the end of the object list. 472 For those familiar with BSDish use of such operations in route 473 sockets, the equivalent translations are: 475 - BSD ADD operation equates to NLM_F_CREATE or-ed 476 with NLM_F_EXCL 477 - BSD CHANGE operation equates to NLM_F_REPLACE 478 - BSD Check operation equates to NLM_F_EXCL 479 - BSD APPEND equivalent is actually mapped to 480 NLM_F_CREATE 482 Sequence Number: 32 bits 483 The sequence number of the message. 485 Process PID: 32 bits 486 The PID of the process sending the message. The PID is used by the 487 kernel to multiplex to the correct sockets. A PID of zero is used 488 when sending messages to user space from the kernel. netlink service 489 fills in an appropiate value when zero. 491 3.3.2.1. Mechanisms for creating protocols 493 One could create a reliable protocol between an FEC and a CPC by 494 using the combination of sequence numbers, ACKs and retransmit 495 timers. Both sequence numbers and sequence numbers are provided by 496 netlink. Timers are provided by Linux. 498 One could create a heartbeat protocol between the FEC and CPC by 499 using the ECHO flags and the NLMSG_NOOP message. 501 3.3.2.2. The ACK netlink message 503 This message is actually used to denote both an ACK and a NACK. 504 Typically the direction is from kernel to user space (in response 505 to an ACK request message that is sent). However, user space should 506 be able to send ACKs back to kernel space when requested. This is 507 IP service specific. 509 jhs_hk_ak_ank draft-forces-netlink-01.txt 511 0 1 2 3 512 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 513 0 1 2 3 514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 | Netlink message header | 516 | type = NLMSG_ERROR | 517 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 518 | error code | 519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 520 | OLD Netlink message header | 521 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 523 Error code: integer (typically 32 bits) 525 Error code of zero indicates that the message is an ACK response. 526 An ACK response message contains the original netlink message 527 header that can be used to compare against (sent sequence numbers 528 etc). 530 A non-zero error message is equivalent to a Negative ACK (NACK). 531 In such a situation, the netlink data that was sent down to the 532 kernel is returned appended to the original netlink message header. 533 An error code printable via the perror() is also set (not in the 534 message header, rather in the executing environment state vari- 535 able). 537 3.3.3. FE services' templates 539 These are services that are offered by the system for general use 540 by other services. They include ability to configure and listen to 541 changes in resource management. IP address management, link events 542 etc fit here. We separate them into this section here for logical 543 purposes despite the fact that they are accessed via the 544 NETLINK_ROUTE FEC. The reason that they exist within NETLINK_ROUTE 545 is due to historical cruft based on the fact that BSD 4.4 rather 546 narrowly focussed Route Sockets implemented them as part of the 547 IPV4 forwarding sockets. 549 3.3.3.1. 551 Network Interface Service Module 553 jhs_hk_ak_ank draft-forces-netlink-01.txt 555 This service provides the ability to create, remove or get informa- 556 tion about a specific network interface. The network interface 557 could be either pohysical or virtual and is network protocol inde- 558 pendent (example an x.25 interface can be defined via this mes- 559 sage). The Interface service message template is shown below. 561 0 1 2 3 562 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 563 0 1 2 3 564 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 565 | Family | Padding | Device Type | 566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 567 | Interface Index | 568 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 569 | Device Flags | 570 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 571 | Change Mask | 572 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 574 Family: This is always set to AF_UNSPEC 576 Device Type: This defines the type of the link. The link could be 577 ethernet, a tunnel etc. Although we are interested only in IPV4, 578 the link type is protocol independent. 580 Interface Index: uniquely identifies interface. 582 Device Flags: 584 IFF_UP Interface is running. 585 IFF_BROADCAST Valid broadcast address set. 586 IFF_DEBUG Internal debugging flag. 587 IFF_LOOPBACK Interface is a loopback interface. 588 IFF_POINTOPOINT Interface is a point-to-point link. 589 IFF_RUNNING Resources allocated. 590 IFF_NOARP No arp protocol 591 IFF_PROMISC Interface is in promiscuous mode. 592 IFF_NOTRAILERS Avoid use of trailers. 593 IFF_ALLMULTI Receive all multicast packets. 594 IFF_MASTER Master of a load balancing bundle. 595 IFF_SLAVE Slave of a load balancing bundle. 596 IFF_MULTICAST Supports multicast 597 IFF_PORTSEL Is able to select media type via ifmap. 598 IFF_AUTOMEDIA Auto media selection active. 599 IFF_DYNAMIC Interface Address is not permanent. 601 jhs_hk_ak_ank draft-forces-netlink-01.txt 603 Change Mask: Reserved for future use. Must be set to 0xFFFFFFFF. 605 Applicable attributes: 606 attribute description 607 ....................................................... 608 IFLA_UNSPEC - unspecified. 609 IFLA_ADDRESS hardware address interface L2 address 610 IFLA_BROADCAST hardware address L2 broadcast 611 address. 612 IFLA_IFNAME ascii string Device name. 613 IFLA_MTU MTU of the device. 614 IFLA_LINK Link type. 615 IFLA_QDISC ascii string defining Queueing disci- 616 pline. 617 IFLA_STATS Interface Statistics. 619 Netlink message types specific to this service: RTM_NEWLINK, 620 RTM_DELLINK, RTM_GETLINK 622 3.3.3.2. IP Address Service module 624 This service provides the ability to add, remove or receive information 625 about an IP address associated with an interface. The Address provi- 626 sioning service message template is shown below. 628 0 1 2 3 629 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 630 0 1 2 3 631 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 632 | Family | Length | Flags | Scope | 633 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 634 | Interface Index | 635 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 Family: AF_INET for IPV4 or AF_INET6 for IPV6. Length: the 638 length of the address mask Flags: IFA_F_SECONDARY for secondary 639 address (old alias interface), 640 IFA_F_PERMANENT for a permanent address set by the user as 641 opposed to dynamic addresses. 642 other flags include: 643 IFA_F_DEPRECATED which defines deprecated (IPV6) address 644 IFA_F_TENTATIVE which defines tentative (IPV6) address 646 Scope: the address scope 648 jhs_hk_ak_ank draft-forces-netlink-01.txt 650 Applicable attributes: 651 attribute description 652 ....................................................... 653 IFA_UNSPEC - unspecified. 654 IFA_ADDRESS raw protocol address of interface 655 IFA_LOCAL raw protocol local address 656 IFA_LABEL ascii string name of the interface 657 reffered to. 658 IFA_BROADCAST raw protocol broadcast address. 659 IFA_ANYCAST raw protocol anycast address 660 IFA_CACHEINFO cacheinfo address information. 662 Define cacheinfo here -- JHS 664 netlink messages specific to this service: RTM_NEWADDR, 665 RTM_DELADDR, RTM_GETADDR 667 4. Sample Protocol for The foo IP service 669 Our proverbial IP service "foo" is used again to demonstrate how 670 one can deploy a simple IP service control using netlink. 672 These steps are continued from the "Sample Service Hierachy" sec- 673 tion. 675 4) query for current config of FE component 677 5) receive response to 4) via channel on 3) 679 6) query for current state of IP service foo 681 7) receive response to 6) via channel on 2) 683 9) register the protocol specific packets you would like the FE to 684 forward to you 686 10) send specific service foo commands and receive responses for them 687 if needed 689 4.1. Interacting with other IP services 691 jhs_hk_ak_ank draft-forces-netlink-01.txt 693 The last diagram shows another control component configuring the 694 same service. In this case, it is a proprietary Command Line Inter- 695 face. The CLI (may or ) may not be using the netlink protocol to 696 communicate to the foo component. If the CLI should issue commands 697 that will affect the policy of the FEC for service "foo" then, then 698 the "foo" CPC is notified. It could then make algorithmic decisions 699 based on this input (example if a policy that foo installed was 700 deleted, there might be need to propagate this to all the peers of 701 service "foo"). 703 5. Currently Defined netlink IP services 705 Although there are many other IP services defined which are using 706 netlink, we will only mention those integrated into the kernel 707 today (kernel version 2.4.6). These are: 709 NETLINK_ROUTE,NETLINK_FIREWALL,NETLINK_ARPD,NETLINK_ROUTE6,NETLINK_IP6_FW 710 NETLINK_TAPBASE,NETLINK_SKIP,NETLINK_USERSOCK. 712 5.1. IP Service NETLINK_ROUTE 714 This service allows CPCs to modify the IPv4 routing table in the 715 Forwarding Engine. It can also be used by CPCs to receive routing 716 updates. 718 5.1.1. Network Route Service Module 720 This service provides the ability to create, remove or receive informa- 721 tion about a network route. The service message template is shown 722 below. 724 jhs_hk_ak_ank draft-forces-netlink-01.txt 726 0 1 2 3 727 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 728 0 1 2 3 729 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 730 | Family | Src length | Dest length | TOS | 731 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 732 | Table ID | Protocol | Scope | Type | 733 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 734 | Flags | 735 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 737 Family: Address family of route. AF_INET for IPV4 and AF_INET6 for 738 IPV6. 740 Src length: prefix length of source 742 Dest length: Prefix length of destination IP address 744 TOS: the 8 bit tos (should be deprecated to make room for DSCP) 746 Table ID: Table identifier. Upto 255 route tables are supported. 747 RT_TABLE_UNSPEC an unspecified routing table 748 RT_TABLE_DEFAULT the default table 749 RT_TABLE_MAIN the main table 750 RT_TABLE_LOCAL the local table 752 The user may assign arbitary values between 753 RT_TABLE_UNSPEC and RT_TABLE_DEFAULT. 755 Protocol: identifies what/who added the route. Described further 756 below. 758 jhs_hk_ak_ank draft-forces-netlink-01.txt 760 protocol Route origin. 761 .............................................. 762 RTPROT_UNSPEC unknown 763 RTPROT_REDIRECT by an ICMP redirect 764 (currently unused) 765 RTPROT_KERNEL by the kernel 766 RTPROT_BOOT during boot 767 RTPROT_STATIC by the administrator 769 Values larger than RTPROT_STATIC are not inter- 770 preted by the kernel, they are just for user infor- 771 mation. They may be used to tag the source of a 772 routing information or to distingush between multi- 773 ple routing daemons. See for 774 the routing daemon identifiers which are already 775 assigned. 777 Scope: Route scope (distance to destination). 778 RT_SCOPE_UNIVERSE global route 779 RT_SCOPE_SITE interior route in the 780 local autonomous system 781 RT_SCOPE_LINK route on this link 782 RT_SCOPE_HOST route on the local host 783 RT_SCOPE_NOWHERE destination doesn't exist 785 The values between RT_SCOPE_UNIVERSE and 786 RT_SCOPE_SITE are available to the user. 788 Type: The type of route. 790 jhs_hk_ak_ank draft-forces-netlink-01.txt 792 Route type description 793 ------------------------------------------------- 794 RTN_UNSPEC unknown route 795 RTN_UNICAST a gateway or direct route 796 RTN_LOCAL a local interface route 797 RTN_BROADCAST a local broadcast route 798 (sent as a broadcast) 799 RTN_ANYCAST a local broadcast route 800 (sent as a unicast) 801 RTN_MULTICAST a multicast route 802 RTN_BLACKHOLE a packet dropping route 803 RTN_UNREACHABLE an unreachable destination 804 RTN_PROHIBIT a packet rejection route 805 RTN_THROW continue routing lookup in another 806 table 807 RTN_NAT a network address translation rule 808 RTN_XRESOLVE refer to an external resolver (not 809 implemented) 811 Flags: further qualify the route. 812 RTM_F_NOTIFY if the route changes, notify the 813 user via rtnetlink 814 RTM_F_CLONED route is cloned from another route 815 RTM_F_EQUALIZE a multicast equalizer (not yet 816 implemented) 818 Attributes applicable to this service: 820 jhs_hk_ak_ank draft-forces-netlink-01.txt 822 Attribute description 823 ----------------------------------------------- 824 RTA_UNSPEC ignored. 825 RTA_DST protocol address for route 826 destination address. 827 RTA_SRC protocol address for route source 828 address. 829 RTA_IIF Input interface index. 830 RTA_OIF Output interface index. 831 RTA_GATEWAY protocol address for the gateway of 832 the route 833 RTA_PRIORITY Priority of route. 834 RTA_PREFSRC 835 RTA_METRICS Route metric 836 RTA_MULTIPATH 837 RTA_PROTOINFO 838 RTA_FLOW 839 RTA_CACHEINFO 841 additional netlink message types applicable to this service: 842 RTM_NEWROUTE, RTM_DELROUTE, RTM_GETROUTE 844 5.1.2. Neighbour Setup Service Module 846 This service provides the ability to add, remove or receive infor- 847 mation about a neighbour table entry (e.g. an ARP entry). The ser- 848 vice message template is shown below. 850 0 1 2 3 851 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 852 0 1 2 3 853 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 854 | Family | Padding | Padding | 855 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 856 | Interface Index | 857 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 858 | State | Flags | Type | 859 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 861 Family: Address Family Interface Index: The unique interface index 862 State: is a bitmask of the following states: 864 jhs_hk_ak_ank draft-forces-netlink-01.txt 866 NUD_INCOMPLETE a currently resolving cache entry 867 NUD_REACHABLE a confirmed working cache entry 868 NUD_STALE an expired cache entry 869 NUD_DELAY an entry waiting for a timer 870 NUD_PROBE a cache entry that is currently 871 reprobed 872 NUD_FAILED an invalid cache entry 873 NUD_NOARP a device with no destination cache 874 NUD_PERMANENT a static entry 876 Flags: one of: 877 NTF_PROXY a proxy arp entry 878 NTF_ROUTER an IPv6 router 880 Attributes applicable to this service: 881 Attribute$ description 882 ------------------------------------ 883 NDA_UNSPEC unknown type 884 NDA_DST a neighbour cache network 885 layer destination address 886 NDA_LLADDR a neighbour cache link layer 887 address 888 NDA_CACHEINFO cache statistics. 890 Describe the NDA_CACHEINFO nda_cacheinfo header later --JHS 892 additional netlink message types applicable to this service: 893 RTM_NEWNEIGH, RTM_DELNEIGH, RTM_GETNEIGH 895 5.1.3. Traffic Control Service 897 This service provides the ability to add, remove or get a queueing dis- 898 cipline. The service message template is shown below. 900 jhs_hk_ak_ank draft-forces-netlink-01.txt 902 0 1 2 3 903 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 904 0 1 2 3 905 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 906 | Family | Padding | Padding | 907 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 908 | Interface Index | 909 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 910 | Qdisc handle | 911 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 912 | Parent Qdisc | 913 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 914 | TCM Info | 915 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 917 5.2. IP Service NETLINK_FIREWALL 919 This service allows CPCs to receive packets sent by the IPv4 fire- 920 wall service in the FE. 922 Two types of messages exist that can be sent from CPC to FEC. These 923 are: Mode messages and Verdict messages. The formats are described 924 below. 926 The Verdict message format is as follows 928 0 1 2 3 929 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 930 0 1 2 3 931 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 932 | Value | 933 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 934 | Packet ID | 935 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 936 | Data Length | 937 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 938 | Payload ... | 939 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 941 A ipq_packet_msg packet type is sent from the FEC to the CPC. The 942 format is described below ==> We need to complete this later 944 jhs_hk_ak_ank draft-forces-netlink-01.txt 946 5.3. IP Service NETLINK_ARPD 948 This service is used by CPCs for managing the ARP table in FE. 950 5.4. IP Service NETLINK_ROUTE6 952 This service allows CPCs to modify the IPv6 routing table in the 953 FE. It can also be used by CPCs to receive routing updates. 955 jhs_hk_ak_ank draft-forces-netlink-01.txt 957 0 1 2 3 958 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 959 0 1 2 3 960 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 961 | IPv6 dst addr | 962 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 963 | IPv6 dst addr | 964 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 965 | IPv6 dst addr | 966 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 967 | IPv6 dst addr | 968 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 969 | IPv6 src addr | 970 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 971 | IPv6 src addr | 972 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 973 | IPv6 src addr | 974 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 975 | IPv6 src addr | 976 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 977 | IPv6 gw addr | 978 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 979 | IPv6 gw addr | 980 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 981 | IPv6 gw addr | 982 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 983 | IPv6 gw addr | 984 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 985 | Type | 986 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 987 | dst length | src length | 988 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 989 | Metric | 990 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 991 | Info | 992 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 993 | Flags | 994 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 995 | Interface Index | 996 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 998 5.5. IP Service NETLINK_IP6_FW 1000 This service allows CPCs to receive packets that failed the IPv6 1002 jhs_hk_ak_ank draft-forces-netlink-01.txt 1004 firewall checks by that module in the FE. 1006 5.6. IP Service NETLINK_TAPBASE 1008 This service allows CPCs to simulate an ethernet driver belonging 1009 to the FE. 1011 //are the instances of the ethertap device. Ethertap //is a 1012 pseudo network tunnel device that allows an //ethernet driver to 1013 be simulated from user space. 1015 5.7. IP Service NETLINK_SKIP 1017 This service is reserved for ENskip (?). 1019 5.8. IP Service NETLINK_USERSOCK 1021 This service is reserved for future Control Plane to FE protocols. 1023 6. Security Considerations 1025 Netlink lives in a trusted environment of a single host separated 1026 by kernel and user space. Linux capabilities ensures that only 1027 someone with CAP_NET_ADMIN capability (typically root user) is 1028 allowed to open sockets. 1030 7. References 1032 [RFC1633] R. Braden, D. Clark, and S. Shenker, "Integrated 1033 Services in the Internet Architecture: an Overview", RFC 1633, 1035 jhs_hk_ak_ank draft-forces-netlink-01.txt 1037 ISI, MIT, and PARC, June 1994. 1039 [RFC1812] F. Baker, "Requirements for IP Version 4 1040 Routers", RFC 1812, June 1995. 1042 [RFC2475] M. Carlson, W. Weiss, S. Blake, Z. Wang, D. 1043 Black, and E. Davies, "An Architecture for Differentiated 1044 Services", RFC 2475, December 1998. 1046 [RFC2748] J. Boyle, R. Cohen, D. Durham, S. Herzog, R. 1047 Rajan, A. Sastry, "The COPS (Common Open Policy Service) Pro- 1048 tocol", RFC 2748, January 2000. 1050 [RFC2328] J. Moy, "OSPF Version 2", RFC 2328, April 1998. 1052 [RFC1157] J.D. Case, M. Fedor, M.L. Schoffstall, C. Davin, 1053 "Simple Network Management Protocol (SNMP)", RFC 1157, May 1054 1990. 1056 [RFC3036] L. Andersson, P. Doolan, N. Feldman, A. Fredette, 1057 B. Thomas "LDP Specification", RFC 3036, January 2001. 1059 [stevens] G.R Wright, W. Richard Stevens. "TCP/IP Illus- 1060 trated Volume 2, Chapter 20", June 1995 1062 8. Acknowledgements 1064 1) Andi Kleen for man pages on netlink and rtnetlink. 1066 2) Alexey Kuznetsov is credited for extending netlink to the IP ser- 1067 vice delivery model. The original netlink character device was 1068 written by Alan Cox. 1070 jhs_hk_ak_ank draft-forces-netlink-01.txt 1072 9. Author's Address: 1074 Jamal Hadi Salim 1075 Znyx Networks 1076 Ottawa, Ontario 1077 Canada 1078 hadi@znyx.com 1080 Hormuzd M Khosravi 1081 Intel 1082 2111 N.E. 25th Avenue JF3-206 1083 Hillsboro OR 97124-5961 1084 USA 1085 1 503 264 0334 1086 hormuzd.m.khosravi@intel.com 1088 Andi Kleen 1089 SuSE 1090 Stahlgruberring 28 1091 81829 Muenchen 1092 Germany