idnits 2.17.1 draft-pinkerton-rddp-security-00.txt: ** The Abstract section seems to be numbered -(176): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(384): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(420): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 43 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack an Authors' Addresses Section. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 248: '...her applications MUST be done using th...' RFC 2119 keyword, line 632: '...and NS-RT, it is RECOMMENDED that each...' RFC 2119 keyword, line 643: '... and S-LT, it is RECOMMENDED that the ...' RFC 2119 keyword, line 673: '... Thus, it is RECOMMENDED that an RI ...' RFC 2119 keyword, line 700: '... It is RECOMMENDED that the Local Pe...' (15 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2003) is 7621 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2828' is defined on line 1274, but no explicit reference was found in the text == Unused Reference: 'DDP' is defined on line 1277, but no explicit reference was found in the text == Unused Reference: 'RDMAP' is defined on line 1281, but no explicit reference was found in the text == Unused Reference: 'SEC-CONS' is defined on line 1285, but no explicit reference was found in the text == Unused Reference: 'RFC2404' is defined on line 1295, but no explicit reference was found in the text == Unused Reference: 'IPv6-Trust' is defined on line 1313, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2828 (Obsoleted by RFC 4949) == Outdated reference: A later version (-07) exists of draft-ietf-rddp-ddp-01 == Outdated reference: A later version (-07) exists of draft-ietf-rddp-rdmap-01 -- No information found for draft-ab-sec-cons - is the name correct? -- Possible downref: Normative reference to a draft: ref. 'SEC-CONS' ** Obsolete normative reference: RFC 2401 (Obsoleted by RFC 4301) ** Obsolete normative reference: RFC 2402 (Obsoleted by RFC 4302, RFC 4305) ** Obsolete normative reference: RFC 2406 (Obsoleted by RFC 4303, RFC 4305) ** Obsolete normative reference: RFC 2407 (Obsoleted by RFC 4306) ** Obsolete normative reference: RFC 2408 (Obsoleted by RFC 4306) ** Obsolete normative reference: RFC 2409 (Obsoleted by RFC 4306) == Outdated reference: A later version (-04) exists of draft-ietf-send-psreq-01 Summary: 13 errors (**), 0 flaws (~~), 12 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft Jim Pinkerton 3 Document: draft-pinkerton-rddp-security-00.txt Microsoft Corporation 4 Expires: December, 2003 Ellen Deleganes 5 Intel Corporation 6 Bernard Aboba 7 Microsoft Corporation 9 June 2003 11 DDP/RDMAP Security 13 1 Status of this Memo 15 This document is an Internet-Draft and is in full conformance with 16 all provisions of Section 10 of RFC2026. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six 24 months and may be updated, replaced, or obsoleted by other documents 25 at any time. It is inappropriate to use Internet-Drafts as 26 reference material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 2 Abstract 36 This document analyzes security issues around implementation and use 37 of the Direct Data Placement Protocol(DDP) and Remote Direct Memory 38 Access Protocol (RDMAP). It first defines an architectural model for 39 an RDMA Network Interface Card (RNIC), which can implement DDP or 40 RDMAP and DDP. The model includes a definition of resources that can 41 be attacked. This document then introduces various Trust Models 42 between a local peer and a remote peer and the tools that can be 43 used to create countermeasures against attacks. Finally, the 44 document reviews various attacks and the countermeasures to be used 45 against them, grouping the attacks into spoofing, tampering, 47 J. Pinkerton, et al. Expires December 2003 1 48 information disclosure, denial of service, and elevation of 49 privilege. 51 Table of Contents 53 1 Status of this Memo.........................................1 54 2 Abstract....................................................1 55 2.1 Issues......................................................4 56 3 Introduction................................................5 57 4 Architectural Model.........................................7 58 4.1 Components..................................................8 59 4.2 Resources...................................................9 60 4.2.1 Connection Context Memory..................................9 61 4.2.2 Data Buffers...............................................9 62 4.2.3 Page Translation Tables...................................10 63 4.2.4 STag Namespace............................................10 64 4.2.5 Completion Queues.........................................10 65 4.2.6 RDMA Read Request Queue...................................10 66 4.3 System Properties..........................................10 67 5 Trust Models...............................................11 68 6 Attacker Capabilities......................................13 69 7 Attacks and Countermeasures................................14 70 7.1 Tools for Countermeasures..................................14 71 7.1.1 Protection Domain (PD)....................................14 72 7.1.2 Limiting STag Scope.......................................14 73 7.1.3 Access Rights.............................................15 74 7.1.4 Limiting the Scope of the Completion Queue................15 75 7.1.5 Limiting the Scope of an Error............................16 76 7.2 Spoofing...................................................16 77 7.2.1 Using an STag on a different connection...................16 78 7.3 Tampering..................................................17 79 7.3.1 RDMA Write or RDMA Read Response to Memory Outside of the 80 Buffer 18 81 7.3.2 Modifying a Buffer After Indicating the Contents Are Ready18 82 7.4 Information Disclosure.....................................19 83 7.4.1 Probing memory outside of the buffer bounds...............19 84 7.4.2 Using RDMA Read to Access Stale Data......................19 85 7.4.3 Accessing a buffer after the transfer is over.............20 86 7.4.4 Accessing data within a valid STag that was unintended....20 87 7.4.5 Using RDMA Read on a buffer meant only for RDMA Write.....20 88 7.4.6 Using Multiple Stags to access the same buffer............21 89 7.4.7 Remote node loading firmware onto the RNIC................21 90 7.5 Denial of Service (DOS)....................................21 91 7.5.1 RNIC Resource Consumption.................................21 92 7.5.2 Resource Consumption By Active Applications...............22 93 7.5.2.1 Receive Data Buffers..................................23 94 7.5.2.2 Completion Queue (CQ) Resource Consumption............24 95 7.5.2.3 RDMA Read Request Queue...............................26 96 7.5.3 Resource Consumption by Idle Applications.................27 97 7.5.4 Exercise of non-optimal code paths........................27 98 7.6 Elevation of Privilege.....................................28 99 7.6.1 Loading Firmware into the RNIC............................28 100 8 IPsec and RDDP.............................................29 101 8.1 Introduction to IPsec......................................29 102 8.2 Recommendations for IPsec Encapsulation of RDDP............30 103 9 Summary Table of Attacks and Trust Models..................31 104 10 References.................................................33 105 10.1 Normative References......................................33 106 10.2 Informative References....................................34 107 11 Author�s Addresses.........................................35 108 12 Acknowledgments............................................36 109 13 Full Copyright Statement...................................37 111 Table of Figures 113 Figure 1 � RDMA Security Model....................................7 114 Figure 2 � Summary Attacks and Trust Model Table.................32 116 2.1 Issues 118 This section is temporary and will go away when all issues have been 119 resolved. 121 Note: this is far from a complete list of issues; as more are 122 raised, they will be added to this list until some sort of consensus 123 is reached. They are in the order found in the specification. 125 Issue: IPsec recommendations for RDMAP/DDP.......................30 127 3 Introduction 129 RDMA enables new levels of flexibility when communicating between 130 two parties compared to current conventional networking practice 131 (e.g. a stream-based model or datagram model). This flexibility 132 brings new security issues that must be carefully understood when 133 designing application protocols utilizing RDMA and when implementing 134 RDMA-aware NICs (RNICs). Note that for the purposes of this security 135 analysis, an RNIC may implement RDMAP and DDP, or just DDP. 137 This specification is work in progress � the intent is to begin the 138 discussion with a well thought out framework to analyze the issues. 139 An area of particular concern is that there may be more attacks 140 possible than have been enumerated here. 142 The specification first develops an architectural model that is 143 relevant for the security analysis � it details components, 144 resources, and system properties that may be attacked. The 145 specification then defines Trust Models. Trust is defined as: 147 Trust - When one party depends upon the other party to not 148 subvert the goals of the protocols, e.g., it will not attempt 149 to perform the following attacks: spoofing, repudiation, 150 information disclosure, denial of service, or elevation of 151 privilege. 153 An Untrusted peer is a party that may (or may not) attempt to 154 perform one or more of the above attacks. A partially trusted peer 155 (either the Local Peer or Remote Peer) may be trusted to not attempt 156 to perform some subset of the above attacks, but not trusted to 157 perform a different subset. 159 For the Untrusted peer, a brief list of capabilities is enumerated. 160 The rest of the specification is focused on analyzing attacks. 161 First, the tools for mitigating attacks are listed, and then a 162 series of attacks on components, resources, or system properties is 163 enumerated. For each attack, possible countermeasures are reviewed. 165 Applications within a host are divided into two categories � 166 Privileged and Non-Privileged. Both application types can send and 167 receive data and request resources. The key differences between the 168 two are: 170 The Privileged Application is partially trusted. It is assumed 171 that the Privileged Application will not intentionally attack 172 the system (e.g., it is a kernel application), although it may 173 be greedy for resources. 175 A Non-Privileged Application�s capabilities are a logical sub- 176 set of the Privileged Application�s. It is assumed by the local 177 host infrastructure that a Non-Privileged Application is 178 Untrusted. All Non-Privileged Application interactions with the 179 RNIC Engine that could affect other applications need to be 180 done through a Trusted intermediary that can verify the Non- 181 Privileged Application requests. 183 4 Architectural Model 185 This section describes an RDMA architectural reference model that is 186 used as security issues are examined. It introduces the components 187 of the model, the resources that can be attacked, and the system 188 properties, which should be preserved when under attack. 190 Figure 1 shows the components comprising the architecture and the 191 interfaces where potential security attacks could be launched. 192 External attacks can be injected into the system from an application 193 that sits above the RI or from the Internet. 195 +-------------+ Request Proxy Interface 196 | Privileged |<--------------------------------+ 197 | Resource | | 198 Admin<-->| Manager | App Control Interface | 199 | |<------+-------------------+ | 200 +-------------+ | | | 201 ^ v v v 202 | +-------------+ +-----------------+ 203 | | Privileged | | Non-Privileged | 204 | | Application | | Application | 205 | +-------------+ +-----------------+ 206 | ^ ^ 207 |Privileged |Privileged |Non-Privileged 208 |Control |Data |Data 209 |Interface |Interface |Interface 210 RNIC | | | 211 Interface(RI) v v v 212 ================================================================= 214 +-----------------------------------------+ 215 | | 216 | RNIC Engine | <------ Firmware 217 | | 218 +-----------------------------------------+ 219 ^ 220 | 221 v 222 Internet 224 Figure 1 � RDMA Security Model 226 4.1 Components 228 The components shown in Figure 1 � RDMA Security Model are: 230 * RNIC Engine � the component that implements the RDMA 231 protocol and/or DDP protocol. 233 * Privileged Resource Manager � the component responsible for 234 managing and allocating resources associated with the RNIC 235 Engine. The Resource Manager does not send or receive data. 237 * Privileged Application � See Section 3 Introduction for a 238 definition of Privileged Application. The local host 239 infrastructure can enable the Privileged Application to map 240 a data buffer directly from the RNIC Engine to the host 241 through the RNIC Interface, but it does not allow the 242 Privileged Application to directly consume RNIC Engine 243 resources. 245 * Non-Privileged Application � See Section 3 Introduction for 246 a definition of Non-Privileged Application. All Non- 247 Privileged Application interactions with the RNIC Engine 248 that could affect other applications MUST be done using the 249 Privileged Resource Manager as a proxy. 251 A design goal of the DDP and RDMAP protocols is to allow, under 252 constrained conditions, Non-Privileged applications to send and 253 receive data directly to/from the RDMA Engine without Privileged 254 Resource Manager intervention - while ensuring that the host remains 255 secure. Thus, one of the primary goals of this paper is to analyze 256 this usage model for the enforcement that is required in the RNIC 257 Engine to ensure the system remains secure. 259 The host interfaces that could be exercised include: 261 * Control Interface - A Privileged Resource Manager uses the 262 RI to allocate and manage RNIC Engine resources, control the 263 state within the RNIC Engine, and monitor various events 264 from the RNIC Engine. It also uses this interface to act as 265 a proxy for some operations that a Non-Privileged 266 Application may require (after performing appropriate 267 countermeasures). 269 * Non-Privileged Data Transfer Interface � A Non-Privileged 270 Application uses this interface to initiate and to check the 271 status of data transfer operations. 273 * Privileged Data Transfer Interface � A superset of the 274 functionality provided by the Non-Privileged Data Transfer 275 Interface. The application is allowed to directly manipulate 276 RNIC Engine mapping resources to map an STag to an 277 application data buffer. 279 * Request Proxy Interface � a Non-Privileged Application uses 280 this interface to control RNIC Engine resources that could 281 affect other applications � such as manipulating the RNIC 282 Engine's mapping of an STag to an application data buffer. 283 The Privileged Resource Manager implements countermeasures 284 to ensure that if the Non-Privileged Application launches an 285 attack it can prevent the attack from affecting other 286 applications. 288 * Figure 1 also shows the ability to load new firmware in the 289 RNIC Engine. Not all RNICs will support this, but it is 290 shown for completeness and is also reviewed under potential 291 attacks. 293 If Internet control messages, such as ICMP, ARP, RIPv4, etc. are 294 processed by the RNIC Engine, the threat analyses for those 295 protocols is also applicable, but outside the scope of this paper. 297 4.2 Resources 299 This section describes the primary resources in the RNIC Engine that 300 could be affected if under attack. For RDMAP, all of the defined 301 resources apply. For DDP, all of the resources except the RDMA Read 302 Queue apply. 304 4.2.1 Connection Context Memory 306 The state information for each connection is maintained in memory, 307 which could be located in a number of places - on the NIC, inside 308 RAM attached to the NIC, in host memory, or in any combination of 309 the three, depending on the implementation. 311 4.2.2 Data Buffers 313 There are two different ways to expose a data buffer; a buffer can 314 be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP 315 Untagged Messages) on DDP Queue zero or the buffer can be exposed 316 for remote access through STags (a.k.a. DDP Tagged Messages). This 317 distinction is important because the attacks and the countermeasures 318 used to protect against the attack are different depending on the 319 method for exposing the buffer to the Internet. 321 4.2.3 Page Translation Tables 323 Page Translation Tables are the structures used by the RNIC to be 324 able to access application memory for data transfer operations. Even 325 though these structures are called "Page" Translation Tables, they 326 may not reference a page at all � conceptually they are used to map 327 an application address space representation of a buffer to the 328 physical addresses that are used by the RNIC Engine to move data. If 329 on a specific system, a mapping is not used, then a subset of the 330 attacks examined may be appropriate. 332 4.2.4 STag Namespace 334 The DDP specification defines a 32-bit namespace for the STag. 335 Implementations may vary in terms of the actual number of STags that 336 are supported. In any case, this is a bounded resource that can come 337 under attack. 339 4.2.5 Completion Queues 341 Completion Queues are used in this specification to conceptually 342 represent how the RNIC Engine notifies the Application of the 343 completion of the transmission of data, or the completion of the 344 reception of data through the Data Transfer Interface. Because there 345 could be many transmissions or receptions in flight at any one time, 346 completions are modeled as a queue rather than a single event. An 347 implementation may also use the Completion Queue to notify the 348 application of other activities, for example, the completion of a 349 mapping of an STag to a specific application buffer. 351 4.2.6 RDMA Read Request Queue 353 The RDMA Read Request Queue is the memory holding state information 354 for one or more RDMA Read Request Messages that have arrived, but 355 for which the RDMA Read Response Messages have not yet been 356 completely sent. 358 4.3 System Properties 360 System properties that can be attacked included system integrity, 361 system stability (liveness, large fluctuations in performance), and 362 confidentiality. 364 5 Trust Models 366 The Trust Models described in this section have three primary 367 distinguishing characteristics. 369 * Local Resource Sharing (yes/no) - When local resources are 370 shared, they are shared across a grouping of RDMAP/DDP 371 Streams. If local resources are not shared, the resources 372 are dedicated on a per Stream basis. Resources are defined 373 in Section 4.2 - Resources on page 9. The advantage of not 374 sharing resources between Streams is that it reduces the 375 number of types of attacks that are possible. The 376 disadvantage is that applications might run out of 377 resources. 379 * Local Trust (yes/no) � Local Trust is determined based on 380 whether the local grouping of RDMAP/DDP Streams (which 381 typically equates to one application or group of 382 applications) mutually trust each other. 384 * Remote Trust (yes/no) � The Remote Trust level is determined 385 based on whether the Local Peer of a specific RDMAP/DDP 386 Stream trusts the Remote Peer of the Stream (using the same 387 definition of trust as stated in the definition of Trust in 388 Section 3 Introduction). 390 It is assumed in this paper that the component that implements the 391 mechanism to control sharing of RNIC Engine resources, is the 392 Privileged Resource Manager. The RNIC Engine exposes its resources 393 through the RI to the Privileged Resource Manager. All Privileged 394 and Non-Privileged applications request resources from the Resource 395 Manager. The Resource Manager implements resource management 396 policies to ensure fair access to resources. The Resource Manager 397 should be designed to take into account security attacks detailed in 398 this specification. 400 The sharing of resources across connections should be under the 401 control of the application, both in terms of the Trust Model the 402 application wishes to operate under, as well as the level of 403 resource sharing the application wishes to give Local Peer 404 processes. 406 Not all of the combinations of the trust characteristics are 407 expected to be used by applications. This paper specifically 408 analyzes five application Trust Models that are expected to be in 409 common use. The Trust Models are as follows: 411 1. NS-NT - Non-Shared Local Resources, no Local Trust, no Remote 412 Trust � typically a server application that wants to run in a 413 mode that has the least number of potential attacks. 415 2. NS-RT - Non-Shared Local Resources, no Local Trust, Remote Trust 416 � typically a peer-to-peer application, which has, by some 417 method outside of the scope of this specification, authenticated 418 the Remote Peer. 420 3. S-NT - Shared Local Resources, no Local Trust, no Remote Trust � 421 typically a server application that runs in an untrusted 422 environment where the amount of resources required is either too 423 large or too dynamic to dedicate for each RDMAP/DDP Stream. 425 4. S-LT - Shared Local Resources, Local Trust, no Remote Trust � 426 typically an application, which provides a session layer and 427 uses multiple Streams, to provide additional throughput or fail- 428 over capabilities. All of the Streams within the local 429 application trust each other, but do not trust the remote peer. 431 5. S-T - Shared Local Resources, Local Trust, Remote Trust � 432 typically a distributed application, such as a distributed 433 database application or a High Performance Computer (HPC) 434 application, which is intended to run on a cluster. Due to 435 extreme resource and performance requirements, the application 436 typically authenticates with all of its peers and then runs in a 437 highly trusted environment. The application peers are all in a 438 single application fault domain and depend on one another to be 439 well-behaved when accessing data structures. If a trusted Remote 440 Peer has an implementation defect that results in poor behavior, 441 the application could be corrupted. 444 Models NS-NT and S-NT above are typical for Internet networking � 445 neither other Local Peers nor the Remote Peer is trusted. Sometimes 446 optimizations can be done that enable sharing of Page Translation 447 Tables across multiple Local Peers, thus Model S-LT can be 448 advantageous. Model S-T is typically used when resource scaling 449 across a large parallel application makes it infeasible to use any 450 other model. Resource scaling issues can either be due to 451 performance around scaling or because there simply are not enough 452 resources. Model NS-RT is probably the least likely model to be 453 used, but is presented for completeness. 455 6 Attacker Capabilities 457 An attacker�s capabilities delimit the types of attacks that 458 attacker is able to launch. RDMAP and DDP require that the initial 459 LLP Stream (and connection) be set up prior to transferring 460 RDMAP/DDP Messages. For the attacker to actively generate an 461 RDMAP/DDP protocol attack, it must have the capability to both send 462 and receive messages. Attackers with send only capabilities should 463 be addressed by the LLP, not by RDMAP/DDP. 465 7 Attacks and Countermeasures 467 This section describes the attacks that are possible against the 468 RDMA system defined in Figure 1 � RDMA Security Model and the RNIC 469 Engine resources defined in Section 4.2. The analysis includes a 470 detailed description of each attack, the Trust Models the attack 471 applies to (see Section 5 for a description of the Trust Models), 472 and a description of the countermeasures appropriate to the Trust 473 Model(s) that can be taken to thwart the attack. 475 Note that, connection setup and teardown is presumed to be done in 476 stream mode (i.e. no RDMA encapsulation of the payload), so there 477 are no new attacks related to connection setup/teardown beyond what 478 is already present in the LLP (e.g. TCP or SCTP). Consequently, any 479 existing analysis of Spoofing, Tampering, Repudiation, Information 480 Disclosure, Denial of Service, or Elevation of Privilege continues 481 to apply. Thus, the analysis in this section focuses on attacks that 482 are present regardless of the LLP Stream type. 484 7.1 Tools for Countermeasures 486 The tools described in this section are the primary mechanisms that 487 can be used to provide countermeasures to potential attacks. 489 7.1.1 Protection Domain (PD) 491 Protection Domains are associated with two of the resources of 492 concern, connection context memory and STags associated with data 493 buffers. Protection Domains are used mainly to ensure that an STag 494 can only be used to access the associated data buffer through 495 connections in the same Protection Domain as that STag. 497 For the Trust Models that are defined to have non-shared resources 498 (Trust Models NS-NT and NS-RT), it is recommended that each Stream 499 be associated with its own, unique Protection Domain. For those 500 Trust Models where resources are shared (Trust Models S-NT, S-LT and 501 S-T), it is recommended that Protection Domain be limited to the 502 number of Streams that share the same Trust Model. 504 7.1.2 Limiting STag Scope 506 The key to protecting a local data buffer is to limit the scope of 507 its STag to the level appropriate for the Trust Model. The scope of 508 the STag can be measured in multiple ways. 510 * Number of Connections and/or Streams on which the STag is 511 valid. One way to limit the scope of the STag is to limit 512 the connections and/or Streams that are allowed to use the 513 STag. As noted in the previous section, use of Protection 514 Domains appropriately can limit the scope of the STag. It is 515 also possible to create an STag that is valid only on a 516 single connection, even in the case where several 517 connections are associated with the Protection Domain of the 518 STag. 520 * Limit the time an STag is valid. By Invalidating an 521 Advertised STag (e.g., revoking remote access to the buffers 522 described by an STag when done with the transfer), an entire 523 class of attacks can be eliminated. 525 * Limit the buffer the STag can reference. Limiting the scope 526 of an STag access to *just* the intended buffers to be 527 exposed is critical to prevent certain forms of attacks. 529 7.1.3 Access Rights 531 Access Rights associated with a specific Advertised STag or 532 RDMAP/DDP Stream provide another mechanism for applications to limit 533 the attack capabilities of the Remote Peer. The Local Peer can 534 control whether a data buffer is exposed for local only, or local 535 and remote access, and assign specific access privileges (read, 536 write, read and write). 538 For DDP, when an STag is advertised, the Remote Peer is presumably 539 given write access rights to the data (otherwise there was not much 540 point to the advertisement). For RDMAP, when an application 541 advertises an STag, it can enable write-only, read-only, or both 542 write and read access rights. 544 Similarly, some applications may wish to provide a single buffer 545 with different access rights on a per-connection or per-Stream 546 basis. For example, some connections may have read-only access, some 547 may have remote read and write access, while on other connections 548 only the Local Peer is allowed access. 550 7.1.4 Limiting the Scope of the Completion Queue 552 Completions associated with sending and receiving data, or setting 553 up buffers for sending and receiving data, could be accumulated in a 554 shared Completion Queue for a group of RDMAP/DDP Streams, or a 555 specific RDMAP/DDP Stream could have a dedicated Completion Queue. 556 Limiting Completion Queue association to one, or a small number of 557 RDMAP/DDP Streams can prevent several forms of Denial of Service 558 attacks. 560 7.1.5 Limiting the Scope of an Error 562 To prevent a variety of attacks, it is important that an RDMAP/DDP 563 implementation be robust in the face of errors. If an error on a 564 specific Stream can cause other unrelated Streams to fail, then a 565 broad class of attacks are enabled against the implementation. 567 7.2 Spoofing 569 Because the RDMAP Stream is only offloaded if it is in the 570 ESTABLISHED state, certain types of traditional forms of wire 571 attacks do not apply -- an end-to-end handshake must have occurred 572 to establish the RDMAP Stream. So, the only form of spoofing that 573 applies is one when a remote node can both send and receive packets. 575 If a man-in-the-middle has the ability to inject packets, which will 576 be accepted by the LLP (e.g., TCP sequence number is correct), one 577 style of attack is for the man-in-the-middle to send Tagged Messages 578 (either RDMAP or DDP). If it can discover a buffer that has been 579 exposed for STag enabled access, then the man-in-the-middle can use 580 an RDMA Read operation to read the contents of the associated data 581 buffer, perform an RDMA Write Operation to modify the contents of 582 the associated data buffer, or invalidate the STag to disable 583 further access to the buffer. The only countermeasure for this form 584 of attack is to either secure the RDMAP/DDP Stream or attempt to 585 provide physical security to prevent man-in-the-middle type access. 587 The best protection against this form of attack is end-to-end 588 authentication, such as IPsec (see Section 8 IPsec and RDDP on page 589 29), to prevent spoofing or tampering. If authentication is not 590 used, then a man-in-the-middle attack can occur, enabling spoofing, 591 tampering, and repudiation. 593 Another approach is to restrict access to only the local 594 subnet/link, and provide some mechanism to limit access, such as 595 physical security or 802.1.x. This model is an extremely limited 596 deployment scenario, and will not be further examined here. 598 7.2.1 Using an STag on a different connection 600 One style of attack from the Remote Peer is for it to attempt to use 601 STag values that it is not authorized to use. Note that, if the 602 Remote Peer sends an invalid STag to the Local Peer, per the DDP and 603 RDMAP specifications, the connection must be torn down. Thus, the 604 threat exists if an STag has been enabled for Remote Access on one 605 connection and a Remote Peer is able to use it on an unrelated 606 connection. If the attack is successful, the attacker could 607 potentially be able to perform either RDMA Read Operations to read 608 the contents of the associated data buffer, perform RDMA Write 609 Operations to modify the contents of the associated data buffer, or 610 to Invalidate the STag to disable further access to the buffer. 612 An attempt by a Remote Peer to access a buffer with an STag on a 613 different connection in the same Protection Domain may or may not be 614 an attack depending on the Trust Model employed by the application. 615 For Trust Model S-T, where resources are shared between connections, 616 and both Local and Remote Peers are Trusted, using an STag on 617 multiple connections within the same Protection Domain is allowed, 618 and could be desired behavior. For the other four Trust Models where 619 the Remote Peer is not Trusted, and/or resources are not intended to 620 be shared, attempting to use an STag on a different connection could 621 be considered to be an attack. 623 In the case where the Trust Model is defined with no shared 624 resources between connections (Trust Models NS-NT and NS-RT), this 625 attack can be defeated by assigning each connection to a different 626 Protection Domain. Before allowing remote access to the buffer, the 627 Protection Domain of the connection where the access attempt was 628 made is matched against the Protection Domain of the STag. If the 629 Protection Domains do not match, access to the buffer is denied, an 630 error is generated, and the RDMAP Stream associated with the 631 attacking connection should be terminated. Thus, for Trust Models 632 NS-NT and NS-RT, it is RECOMMENDED that each connection be in a 633 separate Protection Domain. 635 For Trust Models S-NT and S-LT, where resources are shared, but the 636 Remote Peers are Untrusted, it may not be practical to separate each 637 connection into its own Protection Domain. In this case, the 638 application can still limit the scope of any of the STags it is 639 enabling for remote access to a single connection. If the STag scope 640 has been limited to a single connection, any attempt to use that 641 STag on a different connection will result in an error, and the RDMA 642 Stream associated with that connection should be terminated. Thus, 643 for Trust Models S-NT and S-LT, it is RECOMMENDED that the scope of 644 an STag be limited to a single connection. 646 7.3 Tampering 648 A Remote Peer can attempt to tamper with the contents of data 649 buffers on a Local Peer that have been enabled for remote write 650 access. The types of tampering attacks that are possible are 651 outlined in the sections that follow. 653 7.3.1 RDMA Write or RDMA Read Response to Memory Outside of the Buffer 655 This attack is an attempt by the Remote Peer to perform an RDMA 656 Write or RDMA Read Response to memory outside of the valid length 657 range of the data buffer enabled for remote write access. This 658 attack applies primarily to Trust Models with Untrusted Remote Peers 659 (NS-NT, S-NT and S-LT), and can occur even when no resources are 660 shared across connections. 662 The countermeasure for this type of attack must be in the RNIC 663 implementation, using the STag. When the Local Peer specifies to the 664 RI, the base and the number of bytes in the buffer that it wishes to 665 make accessible, the RI must ensure that the base and bounds check 666 are applied to any access to the buffer referenced by the STag 667 before the STag is enabled for access. When an RDMA data transfer 668 operation (which includes an STag) arrives on a connection, a base 669 and bounds byte granularity access check must be performed to ensure 670 the operation accesses only memory locations within the buffer 671 described by that STag. 673 Thus, it is RECOMMENDED that an RI implementation ensure that a 674 Remote Peer, regardless of Trust Model, will not be able to access 675 memory outside of the buffer specified when the STag was enabled for 676 remote access. 678 7.3.2 Modifying a Buffer After Indicating the Contents Are Ready 680 This attack occurs if a Remote Peer attempts to modify the contents 681 by performing an RDMA Write or an RDMA Read Response after it had 682 indicated to the Local Peer that the data buffer contents were ready 683 for use. 685 This attack applies primarily to the Trust Models where the Remote 686 Peers are not Trusted (Trust Models NS-NT, S-NT and S-LT), and can 687 occur even when no resources are shared across connections. Note 688 that, an error on the part of a Trusted Remote Peer could also 689 result in this problem. 691 The Local Peer can protect itself from this type of attack by 692 revoking remote access when the original data transfer has completed 693 and before it validates the contents of the buffer. The Local Peer 694 can either do this by explicitly revoking remote access rights for 695 the STag when the Remote Peer indicates the operation has completed, 696 or by checking to make sure the Remote Peer Invalidated the STag 697 through the RDMAP Invalidate capability, and if it did not, the 698 Local Peer then explicitly revokes the STag remote access rights. 700 It is RECOMMENDED that the Local Peer follow the above procedure for 701 Trust Models NS-NT, S-NT, and S-LT to protect the buffer. The Local 702 Peer MAY also wish to use this procedure for Trust Models NS-RT and 703 S-T to protect itself from unintended tampering due to an error in 704 the Remote Peer. 706 7.4 Information Disclosure 708 The main potential source for information disclosure is through a 709 local buffer that has been enabled for remote access. If the buffer 710 can be probed by a Remote Peer on another connection, then there is 711 potential for information disclosure. 713 Information disclosure attacks mainly apply to the Trust Models that 714 include Untrusted Remote Peers (Trust Models NS-NT, S-NT, and S-LT 715 as defined in Section 5). Trusted Remote Peers are assumed not to 716 purposely attempt such attacks - any attempt is assumed to be due to 717 an error or other unexpected failure in the Remote Peer. 719 The potential attacks that could result in unintended information 720 disclosure and countermeasures are as follows: 722 7.4.1 Probing memory outside of the buffer bounds 724 This is essentially the same attack as described in Section 7.3.1, 725 except an RDMA Read Request is used to mount the attack. The same 726 countermeasure applies. 728 7.4.2 Using RDMA Read to Access Stale Data 730 If a buffer is being used for a combination of reads and writes 731 (either remote or local), and is exposed to the Remote Peer with at 732 least remote read access rights, the Remote Peer may be able to 733 examine the contents of the buffer before they are initialized with 734 the correct data. In this situation, whatever contents were present 735 in the buffer before the buffer is initialized can be viewed by the 736 Remote Peer, if the Remote Peer performs an RDMA Read. This threat 737 applies to Trust Models NS-NT, S-NT, and S-LT. 739 Because of this, it is RECOMMENDED that the Local Peer ensure that 740 no stale data is contained in the buffer when remote read access 741 rights are initially granted (this can be done by zeroing the 742 contents of the memory, for example). 744 7.4.3 Accessing a buffer after the transfer is over 746 If the Remote Peer has remote read access to a buffer, and by some 747 mechanism tells the Local Peer that the transfer has been completed, 748 but the Local Peer does not disable remote access to the buffer 749 before modifying the data, it is possible for the Remote Peer to 750 retrieve the new data. 752 This is similar to the attack defined in Section 7.3.2 Modifying a 753 Buffer After Indicating the Contents Are Ready on page 18. The same 754 countermeasures apply. In addition, it is RECOMMENDED that the Local 755 Peer should grant remote read access rights only for the amount of 756 time needed to retrieve the data. 758 7.4.4 Accessing data within a valid STag that was unintended 760 762 If the Local Peer enables remote access to a buffer using an STag 763 that references the entire buffer, but intends only a portion of the 764 buffer to be accessed, it is possible for the Remote Peer to access 765 the other parts of the buffer anyway. This threat applies to Trust 766 Models NS-NT, S-NT, and S-LT. 768 To prevent this attack, it is RECOMMENDED that the Local Peer set 769 the base and bounds of the buffer when the STag is initialized to 770 expose only the data to be retrieved. 772 7.4.5 Using RDMA Read on a buffer meant only for RDMA Write 774 One form of disclosure can occur if the access rights on the buffer 775 were set for remote read, when only remote write access was 776 intended. This attack applies primarily to Trust Models with 777 Untrusted Remote Peers (NS-NT, S-NT and S-LT). If the buffer 778 contained application data, or data from a transfer on an unrelated 779 connection, the Remote Peer could retrieve the data through an RDMA 780 Read operation. 782 The most obvious countermeasure for this attack is to not grant 783 remote read access if the buffer is intended to be write-only. The 784 Remote Peer would not be able to retrieve data associated with the 785 buffer. An attempt to do so would result in an error and the RDMAP 786 Stream associated with the connection would be terminated. 788 Thus, it is RECOMMENDED that if an application only intends a buffer 789 to be exposed for remote write access, it set the access rights to 790 the buffer to only enable remote write access. 792 7.4.6 Using Multiple Stags to access the same buffer 794 Multiple STags accessing the same buffer at the same time can result 795 in unintentional information disclosure. When one STag has remote 796 read access enabled and a different STag has remote write access 797 enabled to the same buffer, it is possible for one connection to 798 view the contents that have been written by another Remote Peer. 799 This is not a problem when all of the STags accessing the buffer 800 have only remote read enabled. For Trust Models NS-NT, S-NT, S-LT it 801 is RECOMMENDED that multiple Remote Peers not be granted access to 802 the same buffer through different STags at the same time. A buffer 803 should be exposed to only one Untrusted Remote Peer at a time to 804 ensure that no information disclosure occurs between peers. 806 7.4.7 Remote node loading firmware onto the RNIC 808 If the Remote Peer can cause firmware to be loaded onto the RNIC, 809 there is an opportunity for information disclosure. See Elevation of 810 Privilege in Section 7.6 for this analysis. 812 7.5 Denial of Service (DOS) 814 A DOS attack is one of the primary security risks of RDMAP. This is 815 because RNIC resources are valuable and scarce, and many application 816 environments require communication with Untrusted Remote Peers. If 817 the remote application can be authenticated or encrypted, clearly, 818 the DOS profile can be reduced. For the purposes of this analysis, 819 it is assumed that the RNIC must be able to operate in Untrusted 820 environments, which are open to DOS style attacks. 822 Denial of service attacks against RI resources are not the typical 823 unknown party spraying packets at a random host (such as a TCP SYN 824 attack). Because the connection must be fully established, the 825 attacker must be able to both send and receive messages over that 826 connection, or be able to guess a valid packet on an existing RDMAP 827 Stream. 829 This section outlines the potential attacks and the countermeasures 830 available for dealing with each attack. For each attack, the Trust 831 Model that it applies to is described. 833 7.5.1 RNIC Resource Consumption 835 This section covers attacks that fall into the general category of a 836 Local Peer attempting to unfairly allocate scarce RNIC resources. 837 The Local Peer may be attempting to allocate resources on its own 838 behalf, or on behalf of a Remote Peer. Resources that fall into this 839 category include: Protection Domains, Connection Context Memory, 840 Translation and Protection Tables, and STag namespace. These can be 841 attacks by currently active Local Peers or ones that allocated 842 resources earlier, but are now idle. 844 These attacks generally apply to any Trust Model that includes 845 Untrusted Local Peers (Trust Models NS-NT, NS-RT and S-NT). This 846 type of attack can occur even when resources are not shared across 847 connections. 849 It is RECOMMENDED that the allocation of all scarce resources be 850 placed under the control of a Resource Manager. This allows the 851 Resource Manager to: 853 * prevent a Local Peer from allocating more than its fair 854 share of resources, and 856 * detect if a Remote Peer is attempting to launch a DOS attack 857 by attempting to create an excessive number of connections 858 and take corrective action (such as refusing the request or 859 applying network layer filters against the Remote Peer). 861 Note that, placing scarce resource management under the control of a 862 Resource Manager also prevents other Trusted Local Peers from 863 attempting to allocate more than their fair share of resources. 865 This analysis assumes that the Resource Manager is responsible for 866 handing out Protection Domains, and RNIC implementations will 867 provide enough Protection Domains to allow the Resource Manager to 868 be able to assign a unique Protection Domain for each unrelated, 869 Untrusted Local Peer (for a bounded, reasonable number of Local 870 Peers). This analysis further assumes that the Resource Manager 871 implements policies to ensure that Untrusted Local Peers are not 872 able to consume all of the Protection Domains through a DOS attack. 873 Note that Protection Domain consumption cannot result from a DOS 874 attack launched by a Remote Peer, unless a Local Peer is acting on 875 the Remote Peer�s behalf. 877 7.5.2 Resource Consumption By Active Applications 879 This section describes DOS attacks from Local and Remote Peers that 880 are actively exchanging messages. Attacks on each RDMA NIC resource 881 are examined, including the Trust Models that apply, and the 882 specific countermeasures. Note that, attacks on Connection Context 883 Memory, Page Translation Tables, and STag namespace are covered in 884 Section 7.5.1 RNIC Resource Consumption, so are not included here. 886 7.5.2.1 Receive Data Buffers 888 The Remote Peer can attempt to consume more than its fair share of 889 receive data buffers (Untagged DDP buffers or for RDMAP buffers 890 consumed with Send Type Messages). If resources are not shared 891 across multiple connections (Trust Models NS-NT, NS-RT), then this 892 attack is not possible because the Remote Peer will not be able to 893 consume more buffers than were allocated to the Stream. The worst 894 case scenario is that the Remote Peer can consume more receive 895 buffers than the Local Peer allowed, resulting in no buffers to be 896 available, which would cause the Remote Peer�s connection to the 897 Local Peer to be torn down. 899 If local receive data buffers are shared among multiple Streams and 900 the Remote Peer is not Trusted (Trust Models S-NT, S-LT), then the 901 Remote Peer can attempt to consume more than its fair share of the 902 receive buffers, causing a different Stream to be short of receive 903 buffers, thus possibly causing the other Stream to be torn down. 905 One method the Local Peer could use is to recognize that a Remote 906 Peer is attempting to use more than its fair share of resources and 907 terminate the Stream. However, if the Local Peer is sufficiently 908 slow, it may be possible for the Remote Peer to still mount a denial 909 of service attack. An RNIC Engine implementation that enables a more 910 robust countermeasure is one that provides high and low-water 911 notifications to enable the Local Peer to detect and prevent DOS 912 attacks against shared data buffers. If a low-water notification is 913 enabled, and the Local Peer is able to bound the amount of time that 914 it takes to replenish receive buffers, and the Local Peer maintains 915 statistics to determine which Remote Peer is consuming buffers, then 916 the Local Peer can size the amount of local receive buffers posted 917 on the receive queue such that the low-water notification can arrive 918 before resources are depleted and corrective action can be taken 919 (e.g., terminate the Stream of the attacking Remote Peer). Enabling 920 the high-water notification can help the Local Peer detect a Remote 921 Peer that is launching an attack by sending a large number of out- 922 of-order packets. The notification is generated when more than the 923 specified number of buffers are in process simultaneously on a 924 Stream (i.e., packets have started to arrive for the buffer, but 925 have not yet been delivered to the ULP). 927 A different countermeasure is for the RNIC Engine to provide the 928 capability to limit the Remote Peer�s ability to consume receive 929 buffers on a per Stream basis. Unfortunately this requires a large 930 amount of state to be tracked on a per RNIC basis. 932 Thus, if an RNIC Engine provides the ability to share receive 933 buffers across multiple Streams, it is RECOMMENDED that it enable 934 the Local Peer to detect if the Remote Peer is attempting to consume 935 more than its fair share of resources so that the application can 936 apply countermeasures to detect and prevent the attack. 938 7.5.2.2 Completion Queue (CQ) Resource Consumption 940 DOS attacks against the Completion Queue can be caused by either the 941 Local Peer or the Remote Peer if either attempts to cause more 942 completions than its fair share, thus potentially starving another 943 unrelated Stream such that no Completion Queue entries are 944 available. 946 A Completion Queue entry can potentially be consumed by a completion 947 from the send queue or a receive completion. In the former, the 948 attacker is the Local Peer (Trust Models S-NT). In the later, the 949 attacker is the Remote Peer (S-NT, S-LT). 951 The potential attacks and the countermeasures for each are described 952 in the subsections that follow. 954 7.5.2.2.1 Local Peer Attacking a Shared CQ 956 A form of attack can occur for Trust Models NS-NT, NS-RT, and S-NT, 957 where the Local Peers are Untrusted, and Local Peers can consume 958 resources on the CQ. Sharing a CQ across connections that belong to 959 different Protection Domains is NOT RECOMMENDED in cases where any 960 of the Local Peers are Untrusted. A Local Peer that is slow to free 961 resources on the CQ by not reaping the completion status quick 962 enough could stall all other Local Peers attempting to use that CQ. 964 One of two countermeasures can be used to avoid this kind of attack. 965 The first is to use a different CQ per Untrusted Local Peer. The 966 other is to use a Trusted Local Peer to act as a third party to free 967 resources on the CQ and place the status in intermediate storage 968 until the Untrusted Local Peer reaps the status information. 970 7.5.2.2.2 Remote Peer Attacking a Shared CQ 972 The Remote Peer can attack a CQ by consuming more than its fair 973 share of CQ entries by using one of two methods. The first method 974 can only be used if the ULP protocol allows the Remote Peer to 975 reserve a specified number of CQ entries, possibly leaving 976 insufficient entries for other connections that are sharing the CQ. 977 The other method is if the Remote Peer can attack the CQ by 978 overwhelming the CQ with completions, which can affect completion 979 processing on other Streams sharing that connection. 981 The first method of attack can be avoided if the ULP does not allow 982 a Remote Peer to reserve CQ entries. This is RECOMMENDED 983 particularly for Trust Models S-NT and S-LT, with shared resources 984 and Untrusted Remote Peers. If a Local Peer allows this type of 985 resource allocation, and it has any Untrusted Remote Peers, then the 986 Local Peer it is RECOMMENDED that the CQ be isolated to connections 987 within a single Protection Domain. 989 One way that a Remote Peer can attempt to overwhelm its CQ with 990 completions is by sending minimum length RDMAP/DDP Messages to cause 991 as many completions per second as possible. Assuming that the Local 992 Peer does not run out of receive buffers (if they do, then this is a 993 different attack, documented in Section 7.5.2.1 Receive Data Buffers 994 on page 23), then it might be possible for the Remote Peer to 995 consume more than its fair share of Completion Queue entries. 996 Depending upon the CQ implementation, this could either cause the CQ 997 to overflow (if it is not large enough to handle all of the 998 completions generated) or for another Stream to not be able to 999 generate CQ entries (if the RNIC had flow control on generation of 1000 CQ entries into the CQ). In either case, the CQ will stop 1001 functioning correctly and any connections expecting completions on 1002 the CQ will stop functioning. 1004 This attack can occur regardless of whether all of the connections 1005 associated with the CQ are in the same Protection Domain or are in 1006 different Protection Domains. Because this attack assumes a shared 1007 local resource and an Untrusted Remote Peer, Trust Models S-NT, S-LT 1008 apply. 1010 The Local Peer can protect itself from this type of attack using 1011 either of the following methods: 1013 * Resize the CQ to the appropriate level(note that resizing 1014 the CQ can fail, so the CQ resize should be done before 1015 sizing the buffers on the connection), OR 1017 * Grant fewer resources than the Remote Peer requested (not 1018 supplying the number of Receive Data Buffers requested). 1020 The proper sizing of the CQ is dependent on the Trust Model. For the 1021 Trust Model described in Section 5, with Trusted Local Peers and 1022 Untrusted Remote Peers (Trust Model S-LT), a correctly sized CQ 1023 means that the CQ is large enough to hold completion status for all 1024 of the outstanding Receive Data Buffers, or: 1026 CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ) 1027 + SUM(MaxPostedOnEachS-RQ) 1028 + SUM(MaxPostedOnEachSQ) 1030 If the Trust Model assumes neither the Local Peer nor the Remote 1031 Peer is trusted (Trust Model S-NT or S-LT), then the CQ must be 1032 sized to accommodate the maximum number of operations or Receive 1033 Data Buffers that it is possible to post at any one time, thus the 1034 equation becomes: 1036 CQ_MIN_SIZE = SUM(SizeOfEachRQ) 1037 + SUM(SizeOfEachS-RQ) 1038 + SUM(SizeOfEachSQ) 1040 Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per connection 1041 or per Shared Receive Queue basis. 1043 7.5.2.3 RDMA Read Request Queue 1045 Two types of attacks are possible against resources associated with 1046 RDMA Read Request Queues. One style of attack can only occur when 1047 the RDMA Read Request Queue resources are pooled across multiple 1048 connections. This attack occurs when an Untrusted Local Peer 1049 attempts to unfairly allocate RDMA Read Request Queue resources for 1050 its connections. 1052 It is RECOMMENDED that access to interfaces that allocate RDMA Read 1053 Request Queue entries be restricted to a Trusted Local Peer, such as 1054 a Resource Manager. The Resource Manager should prevent a Local Peer 1055 from allocating more than its fair share of resources. 1057 Another form of attack is the Remote Peer sending more RDMA Read 1058 Requests than the depth of the RDMA Read Request Queue at the Local 1059 Peer. 1061 This attack can be prevented by properly configuring the connection 1062 when the connection is established. The Remote Peer�s end of the 1063 connection should be configured by a trusted agent such that the 1064 RNIC will not transmit RDMA Read Requests that exceed the depth of 1065 the RDMA Read Request Queue at the Local Peer. If the connection is 1066 correctly configured, and if the Remote Peer submits more requests 1067 than the Local Peer�s RDMA Read Request Queue can handle, the 1068 requests will be queued at the Remote Peer�s connection until 1069 previous requests complete. If the Remote Peer�s connection is not 1070 configured correctly, the RDMAP Stream for that connection is 1071 terminated when more RDMA Read Requests arrive at the Local Peer 1072 than the Local Peer can handle. Thus, the Remote Peer is able to 1073 only affect its own connection. 1075 7.5.3 Resource Consumption by Idle Applications 1077 The simplest form of a DOS attack given a fixed amount of resources 1078 is for the Remote Peer to create a RDMAP Stream to a Local Peer, 1079 request dedicated resources then do no actual work. This allows the 1080 Remote Peer to be very light weight (i.e. only negotiate resources, 1081 but do no data transfer) and consumes a disproportionate amount of 1082 resources in the server. 1084 A general countermeasure for this style of attack is to monitor 1085 active RDMAP Streams and if resources are getting low, reap the 1086 resources from RDMAP Streams that are not transferring data and 1087 possibly terminate the connection. This needs to be under 1088 administrative control, and demonstrates the need for a MIB for 1089 RDMAP so this condition can be detected and acted upon. 1091 Refer to Section 7.5.1 for the analysis and countermeasures for this 1092 style of attack on the following RNIC resources: Connection Context 1093 Memory, Page Translation Tables and STag namespace. 1095 Note that some RNIC resources are not at risk of this type of attack 1096 from a Remote Peer because an attack requires the Remote Peer to 1097 send messages in order to consume the resource. Receive Data 1098 Buffers, Completion Queue, and RDMA Read Request Queue resources are 1099 examples. These resources are, however, at risk form a Local Peer 1100 that attempts to allocate resources, then goes idle. The general 1101 countermeasure described in this section can be used to free 1102 resources allocated by an idle Local Peer. 1104 7.5.4 Exercise of non-optimal code paths 1106 Another form of DOS attack is to attempt to exercise data paths that 1107 can consume a disproportionate amount of resources. An example might 1108 be if error cases are handled on a �slow path� (consuming either 1109 host or RNIC computational resources), and an attacker generates 1110 excessive numbers of errors in an attempt to consume these 1111 resources. 1113 It is RECOMMENDED that an implementation provide the ability to 1114 detect the above condition and allow an administrator to act, 1115 including potentially administratively tearing down the RDMAP Stream 1116 associated with the connection exercising data paths consuming a 1117 disproportionate amount of resources. 1119 7.6 Elevation of Privilege 1121 The RDMAP/DDP Security Architecture explicitly differentiates 1122 between three levels of privilege � Non-Privileged, Privileged, and 1123 the Privileged Resource Manager. If a Non-Privileged Application is 1124 able to elevate its privilege level to a Privileged Application, 1125 then mapping a physical address list to an STag can provide local 1126 and remote access to any physical address location on the node. If a 1127 Privileged Mode Application is able to promote itself to be a 1128 Resource Manager, then it is possible for it to perform denial of 1129 service type attacks where substantial amounts of local resources 1130 could be consumed. 1132 There is only one mechanism discovered to date, other than 1133 implementation defects, which would potentially allow an elevation 1134 of privilege. 1136 7.6.1 Loading Firmware into the RNIC 1138 If the RI implementation, by some insecure mechanism (or 1139 implementation defect), can enable a Remote Peer or un-trusted Local 1140 Peer to load firmware into the RNIC Engine, it is possible to use 1141 the RNIC to attack the host. Thus, it is RECOMMENDED that an 1142 implementation not enable firmware to be loaded on the RNIC Engine 1143 directly from a Remote Peer, unless the update is done via a secure 1144 protocol, such as IPsec (See Section 8 IPsec and RDDP on page 29). 1145 It is RECOMMENDED that an implementation not allow a Non-Privileged 1146 Local Peer to update firmware in the RNIC Engine. 1148 8 IPsec and RDDP 1150 8.1 Introduction to IPsec 1152 IPsec is a protocol suite which is used to secure communication at 1153 the network layer between two peers. The IPsec protocol suite is 1154 specified within the IP Security Architecture [RFC2401], IKE 1155 [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec 1156 Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is 1157 the key management protocol while AH and ESP are used to protect IP 1158 traffic. 1160 An IPsec SA is a one-way security association, uniquely identified 1161 by the 3-tuple: Security Parameter Index (SPI), protocol (ESP) and 1162 destination IP. The parameters for an IPsec security association 1163 are typically established by a key management protocol. These 1164 include the encapsulation mode, encapsulation type, session keys and 1165 SPI values. 1167 IKE is a two phase negotiation protocol based on the modular 1168 exchange of messages defined by ISAKMP [RFC2408],and the IP Security 1169 Domain of Interpretation (DOI) [RFC2407]. IKE has two phases, and 1170 accomplishes the following functions: 1172 1. Protected cipher suite and options negotiation - using keyed 1173 MACs and encryption and anti-replay mechanisms 1175 2. Master key generation - such as via MODP Diffie-Hellman 1176 calculations 1178 3. Authentication of end-points 1180 4. IPsec SA management (selector negotiation, options negotiation, 1181 create, delete, and rekeying) 1183 Items 1 through 3 are accomplished in IKE Phase 1, while item 4 is 1184 handled in IKE Phase 2. 1186 An IKE Phase 2 negotiation is performed to establish both an inbound 1187 and an outbound IPsec SA. The traffic to be protected by an IPsec 1188 SA is determined by a selector which has been proposed by the IKE 1189 initiator and accepted by the IKE Responder. In IPsec transport 1190 mode, the IPsec SA selector can be a "filter" or traffic classifier, 1191 defined as the 5-tuple: . 1194 The successful establishment of a IKE Phase-2 SA results in the 1195 creation of two uni-directional IPsec SAs fully qualified by the 1196 tuple . 1198 The session keys for each IPsec SA are derived from a master key, 1199 typically via a MODP Diffie-Hellman computation. Rekeying of an 1200 existing IPsec SA pair is accomplished by creating two new IPsec 1201 SAs, making them active, and then optionally deleting the older 1202 IPsec SA pair. Typically the new outbound SA is used immediately, 1203 and the old inbound SA is left active to receive packets for some 1204 locally defined time, perhaps 30 seconds or 1 minute. 1206 8.2 Recommendations for IPsec Encapsulation of RDDP 1208 Issue: IPsec recommendations for RDMAP/DDP 1210 This is work that is still to be done. 1212 9 Summary Table of Attacks and Trust Models 1214 1217 Rows are the attack (grouped into categories) 1219 Columns are the: 1221 * Sec � Section the attack is discussed 1223 * Attack Name � short name for the attack 1225 * Threat - threat type (DOS, etc) 1227 * Columns labeled 1-5 are the Trust Model number (see section 1228 5 Trust Models on page 11). Each entry has a value of +, -, 1229 and R (research). 1231 +-------+--------------------------+-------+---+---+---+---+---+ 1232 | Sec | Attack Name |Threat | 1 | 2 | 3 | 4 | 5 | 1233 +-------+--------------------------+-------+---+---+---+---+---+ 1234 | 7.2.1 | STag use on different | Spoof | | | | | | 1235 | | connection in same PD | | | | | | | 1236 +-------+--------------------------+-------+---+---+---+---+---+ 1237 | 7.3.1 | Memory write outside of | Tamper| | | | | | 1238 | | buffer range | | | | | | | 1239 | 7.3.2 | Modify Buffer after | Tamper| | | | | | 1240 | | contents ready | | | | | | | 1241 +-------+--------------------------+-------+---+---+---+---+---+ 1242 | 7.4.1 | Probe memory outside of | ID | | | | | | 1243 | | buffer bounds | | | | | | | 1244 | 7.4.2 | Access stale data | ID | | | | | | 1245 | 7.4.3 | Access buffer after | ID | | | | | | 1246 | | transfer over | | | | | | | 1247 | 7.4.4 | Unintended data access | ID | | | | | | 1248 | | using valid STag | | | | | | | 1249 | 7.4.5 | Using RDMA Read on a | ID | | | | | | 1250 | | buffer meant only for | | | | | | | 1251 | | RDMA Write | | | | | | | 1252 | 7.4.6 | Using multiple STags to | ID | | | | | | 1253 | | access the same buffer | | | | | | | 1254 | 7.4.7 | Remote node loading | ID | | | | | | 1255 | | firmware onto RNIC | | | | | | | 1256 +-------+--------------------------+-------+---+---+---+---+---+ 1257 | 7.5.1 | RNIC resource consumption| DOS | | | | | | 1258 | 7.5.2 | Resource consumption by | DOS | | | | | | 1259 | | active processes | | | | | | | 1260 | 7.5.3 | Resource consumption by | DOS | | | | | | 1261 | | idle processes | | | | | | | 1262 | 7.5.4 | Non-optimal code paths | DOS | | | | | | 1263 +-------+--------------------------+-------+---+---+---+---+---+ 1264 | 7.6.1 | Loading firmware onto | Elev | | | | | | 1265 | | RNIC | | | | | | | 1266 +-------+--------------------------+-------+---+---+---+---+---+ 1268 Figure 2 � Summary Attacks and Trust Model Table 1270 10 References 1272 10.1 Normative References 1274 [RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC 1275 2828, May 2000. 1277 [DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct Data 1278 Placement over Reliable Transports", Internet-Draft draft-ietf- 1279 rddp-ddp-01.txt, February 2003. 1281 [RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA 1282 Protocol Specification", Internet-Draft draft-ietf-rddp-rdmap- 1283 01.txt, February 2003. 1285 [SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing 1286 RFC Text on Security Considerations", Internet-Draft draft-ab- 1287 sec-cons-03.txt, January 2003. 1289 [RFC2401] Atkinson, R. and Kent, S., "Security Architecture for the 1290 Internet Protocol", RFC 2401, November 1998 1292 [RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC 1293 2402, November 1998 1295 [RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within 1296 ESP and AH", RFC 2404, November 1998 1298 [RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security 1299 Payload (ESP)", RFC 2406, November 1998 1301 [RFC2407] Piper, D., "The Internet IP Security Domain of 1302 Interpretation of ISAKMP", RFC 2407, November 1998 1304 [RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J., 1305 "Internet Security Association and Key Management Protocol 1306 (ISAKMP), RFC 2408, November 1998 1308 [RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange 1309 (IKE)", RFC 2409, November 1998 1311 10.2 Informative References 1313 [IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor 1314 Discovery trust modelsTrust Models and threats", Internet-Draft 1315 draft-ietf-send-psreq-01.txt, January 2003. 1317 11 Author�s Addresses 1319 James Pinkerton 1320 Microsoft Corporation 1321 One Microsoft Way 1322 Redmond, WA. 98052 USA 1323 Phone: +1 (425) 705-5442 1324 Email: jpink@windows.microsoft.com 1326 Ellen Deleganes 1327 Intel Corporation 1328 MS JF5-355 1329 2111 NE 25th Ave. 1330 Hillsboro, OR 97124 USA 1331 Phone: +1 (503) 712-4173 1332 Email: ellen.m.deleganes@intel.com 1334 Bernard Aboba 1335 Microsoft Corporation 1336 One Microsoft Way 1337 Redmond, WA. 98052 USA 1338 Phone: +1 1339 Email: 1341 12 Acknowledgments 1343 Allyn Romanow 1345 Catherine Meadows 1347 Pat Thaler 1349 James Livingston 1351 John Carrier 1353 13 Full Copyright Statement 1355 Copyright (C) The Internet Society (2001). All Rights Reserved. 1357 This document and translations of it may be copied and furnished to 1358 others, and derivative works that comment on or otherwise explain it 1359 or assist in its implementation may be prepared, copied, published 1360 and distributed, in whole or in part, without restriction of any 1361 kind, provided that the above copyright notice and this paragraph 1362 are included on all such copies and derivative works. However, this 1363 document itself may not be modified in any way, such as by removing 1364 the copyright notice or references to the Internet Society or other 1365 Internet organizations, except as needed for the purpose of 1366 developing Internet standards in which case the procedures for 1367 copyrights defined in the Internet Standards process must be 1368 followed, or as required to translate it into languages other than 1369 English. 1371 The limited permissions granted above are perpetual and will not be 1372 revoked by the Internet Society or its successors or assigns. 1374 This document and the information contained herein is provided on an 1375 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1376 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1377 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1378 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 1379 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1381 Funding for the RFC Editor function is currently provided by the 1382 Internet Society.