idnits 2.17.1 draft-fiorelli-weirds-rws-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == It seems as if not all pages are separated by form feeds - found 0 form feeds but 9 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 9, 2011) is 4704 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC3986' is defined on line 315, but no explicit reference was found in the text == Unused Reference: 'RFC3987' is defined on line 319, but no explicit reference was found in the text == Unused Reference: 'RFC3707' is defined on line 333, but no explicit reference was found in the text == Unused Reference: 'RFC3912' is defined on line 336, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Fiorelli 3 Internet-Draft RIPE 4 Intended status: Informational June 9, 2011 5 Expires: December 11, 2011 7 The RIPE Database REST API 8 draft-fiorelli-weirds-rws-00 10 Abstract 12 This document describes the RIPE Database REST API. 14 The purpose of this document is to facilitate discussion and serve as 15 input into a standards process in this area, currently being 16 discussed on the WHOIS-based Extensible Internet Registration Data 17 Service (WEIRDS) mailing list 18 (https://www.ietf.org/mailman/listinfo/weirds). 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on December 11, 2011. 37 Copyright Notice 39 Copyright (c) 2011 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 1.1. Why REST? . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 2. Some features of the Query Services . . . . . . . . . . . . . . 5 57 3. Some sample invocation of the query services . . . . . . . . . 6 58 4. Some details on the CRUD Services . . . . . . . . . . . . . . . 6 59 5. Internationalization Considerations . . . . . . . . . . . . . . 7 60 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 7 61 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 8 62 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 63 8.1. Normative References . . . . . . . . . . . . . . . . . . . 8 64 8.2. Informative References . . . . . . . . . . . . . . . . . . 8 65 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 8 67 1. Introduction 69 This document describes a pilot service for querying and displaying 70 data in the RIPE database. The service is implemented using the HTTP 71 protocol [RFC2616] and conforms to the architectural constraints of 72 REST [REST]. In this document, we explain our reasons for new 73 interfaces (and limits of existing ones) as well as high level 74 functionalities of the API. We did not include technical details of 75 our implementation, which can be found as a separate document and is 76 available at: 77 http://labs.ripe.net/Members/bfiorell/api-documentation. 79 1.1. Why REST? 81 The RIPE Database has a large number of power users: organisations 82 and individuals that make or maintain software which depends on the 83 RIPE Database. The way these specialized clients interact with the 84 RIPE Database is dictated by historical and technical aspects of RPSL 85 and the Whois protocol. 87 RPSL ([RFC2622]) is the routing policy specification language, but 88 for historical and convenience reasons it actually specifies much 89 more than routing policies, there are many RFCs related to the RPSL 90 (see [RFC2622], [RFC2650], [RFC4012]) and in practice the language 91 describes more than 20 different types of RPSL objects, it is more a 92 formalisation of the way policy records have been stored and 93 exchanged in the existing Whois systems since to the late 80's 94 instead of a high level domain language specification. 96 The result is that the policy specification language and its 97 extensions are tightly coupled with the way policies are stored in 98 the existing Whois systems and vice versa, so any new system that is 99 implementing the language ends up reproducing a Whois system. 101 The qualities of RPSL and Whois are well known. In this section we 102 highlight their limits by describing the way a client would interact 103 with the two Whois interfaces of query and update. 105 First scenario, a client needs to extract sensible information from 106 one or more Internet Registries like the RIPE Database, the steps 107 are: 109 prepare one or more queries and invoke a command line Whois client 110 program or open a TCP socket connection to one or more Whois 111 servers, 113 parse and post-process the RPSL responses, facing many intricacies 114 like continuation lines, end of line comments, comma separated 115 values, attribute values that can reference to different types of 116 RPSL objects depending on various conditions, etc., 118 if object references need to be resolved, extract referenced 119 primary keys, execute other queries, parse again RPSL to identify 120 the right referenced objects; this step can be recursive, 122 merge results from multiple queries, maybe queries from multiple 123 internet registries that may adopt different RPSL flavours, 125 extract subsets of objects, by applying various criteria, 127 extract subsets of attributes from set of objects, by applying 128 various criteria. 130 The other standard scenarios is that of update: 132 in case of update or delete, fetch the object, this operation may 133 require several of the steps described in the previous workflow, 135 in case of a new object, build RPSL text, maybe starting 136 converting business data into RPSL text, 138 decorate the object with authorisation credentials, adding 139 password hashes to the RPSL text or signing the RPSL text with a 140 PGP key, 142 send the text data via mail to the Mail Update interface or via 143 HTTP post to the Syncupdates interface, 145 in case of error, parse a human readable text response if specific 146 error conditions must be handled in a special way. 148 As shown above, such processes are modeled on a human centered 149 workflow, they are not the optimal to build new services, extend 150 existing ones or build tools on top of them. 152 Concretely, this means clients have to handle too much complexity, 153 many clients have problems at handling all the complexity of RPSL, 154 many are too expensive to be extended and difficult to maintain. 156 RPSL and Whois are still invaluable but it's evident that there is a 157 need for simpler interfaces and machine ready data formats in order 158 to simplify the development of services and tools and increase the 159 value of Registries by exposing new domain specific interfaces. 161 The first step can be the identification of a layer of Service 162 Provider Interfaces and a data schema simple enough to be agnostic of 163 the underlying Registry implementation. 165 The SPI would sit on top of a Registry Database allowing the 166 composition of domain specific Services but also making real-time 167 interoperation between Registries and services on multiple 168 authoritative Databases possible to achieve. 170 With these forces and this vision the RIPE Database Group has been 171 prototyping the RIPE Whois RESTful Query and CRUD API and the related 172 XML schemas. 174 The choice of REST has been quite obvious, HTTP is nowadays the most 175 accessible protocol and it represents a good architectural blueprint 176 for stateless services, with its different HTTP methods, the resource 177 locator protocol, the HTTPS, etc. 179 For the representation schema we have designed a very relaxed 180 attribute oriented XML Schema. We only apply a structural validation 181 via XSD, after some testing we decided to remove any form of 182 attribute or type validation in order to reuse the same schema on 183 different RPSL flavors. 185 The services support representational styles JSON, HTML and plain 186 text, all derived via XSL transformation, with the latter two to 187 showcase the transformation powers of XML and also to demo resource 188 navigation via lookup references on any HTML browser. 190 Content negotiation can be done using HTTP headers or appending a 191 file extension to the request URL. 193 The query services can be used on any RPSL based Whois server or 194 mirror. 196 For example in one request it is possible to execute the same Lookup 197 or Search request on all the Regional Internet Registries and return 198 all the responses as a unique set of resources. 200 Similarly can be done for the CRUD interfaces, building adapter 201 modules for the different update mechanisms provided by different 202 Registries given that they adopt the same set of resource types. 204 Actually we implemented an adapter for the Whois Syncupdates 205 interface in the form of an HTTP proxy. 207 2. Some features of the Query Services 208 Single resource Lookup Service, given a primary key a type and a 209 registry it always return one and only one object. It can also be 210 used to identify resources by URL bookmark. 212 Resolution of referenced resources. Given a resource, all the 213 attribute values that represent references to other resource 214 contain an xlink anchor that can be followed to navigate and 215 browse networks of resources. 217 HATEOAS, the client can navigate through any network of resources 218 via xlinks without requiring any stateful information to be stored 219 on the servers, this comes as a benefit of the two previous 220 features. 222 Normalisation of comma separated values, when they represent 223 references to multiple resources. 225 Normalisation of continuation lines, end of line comments and 226 other RPSL intricacies. This is still a work in progress and 227 still not complete because is difficult to test all the side 228 effects of this operation. The only real solution would be to 229 deprecate continuation lines in RPSL and split them into multiple 230 attribute value pairs. 232 All the Query Services are available also on HTTPS. 234 3. Some sample invocation of the query services 236 A Lookup Service URL for an organization object: 237 http://apps.db.ripe.net/whois/lookup/ripe/organisation/ORG-BME1-RIPE 239 A Lookup Service URL for a route object: 240 http://apps.db.ripe.net/whois/lookup/ripe/route/193.6.23.0/24/AS2547 242 A Search Service URL on multiple registries: http://apps.db.ripe.net/ 243 whois/ 244 search.xml?flags=r&source=ripe&source=apnic&source=afrinic& 245 query-string=AS2547 247 4. Some details on the CRUD Services 249 The CRUD Services are an attempt to split the single overloaded 250 generic update interface provided by Mail Updates and Syncupdates 251 into separate low level interfaces, each defining a simpler contract, 252 designed for programmatic use rather than for human interaction. 254 For example the existing update interfaces always require clients to 255 provide the entire RPSL resource in exactly the same text-format as 256 it is stored in the Whois database, even if the requested operation 257 is just a deletion. A client will have to fetch the RPSL object from 258 the RIPE Database doing an appropriate query, than extract ony the 259 text blob that represent the resource to be deleted and finally it 260 will have to execute a request to the update service providing the 261 entire text and adding delete attributes and authorisation 262 credentials. 264 The new CRUD Services on the other hand provide a Delete interface 265 that only requires a primary key, a type, a registry identifier and 266 one or more passwords. It is the equivalent of a lookup but is 267 executed with the HTTP Delete method. It is exposed only on HTTPS. 269 The client will just have to send a request like: HTTP DELETE: https: 270 //lab.db.ripe.net/whois/delete/test/person/pp16-test?password=123 : 272 The server will just respond with an HTTP Status code indicating 273 success or failure, in case of failure different status codes will 274 indicate different error conditions. 276 On top of Create, Update and Delete methods we also prototyped some 277 attribute modification services that in one only request can 278 implement complex update workflow. 280 For example with one only HTTP request will be possible to execute 281 commands like: 283 replace all the attributes of a given type with a new set of 284 attribute, 286 remove all the attributes that have value matching the given 287 regular expression, 289 add a new set of attributes after the Nth attribute of type X. 291 5. Internationalization Considerations 293 TBA 295 6. IANA Considerations 297 TBA 299 7. Security Considerations 301 TBA. 303 8. References 305 8.1. Normative References 307 [REST] Fielding, R. and R. Taylor, "Principled Design of the 308 Modern Web Architecture", ACM Transactions on Internet 309 Technology Vol. 2, No. 2, May 2002. 311 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 312 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 313 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 315 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 316 Resource Identifier (URI): Generic Syntax", STD 66, 317 RFC 3986, January 2005. 319 [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource 320 Identifiers (IRIs)", RFC 3987, January 2005. 322 8.2. Informative References 324 [RFC2622] Alaettinoglu, C., Villamizar, C., Gerich, E., Kessens, D., 325 Meyer, D., Bates, T., Karrenberg, D., and M. Terpstra, 326 "Routing Policy Specification Language (RPSL)", RFC 2622, 327 June 1999. 329 [RFC2650] Meyer, D., Schmitz, J., Orange, C., Prior, M., and C. 330 Alaettinoglu, "Using RPSL in Practice", RFC 2650, 331 August 1999. 333 [RFC3707] Newton, A., "Cross Registry Internet Service Protocol 334 (CRISP) Requirements", RFC 3707, February 2004. 336 [RFC3912] Daigle, L., "WHOIS Protocol Specification", RFC 3912, 337 September 2004. 339 [RFC4012] Blunk, L., Damas, J., Parent, F., and A. Robachevsky, 340 "Routing Policy Specification Language next generation 341 (RPSLng)", RFC 4012, March 2005. 343 Author's Address 345 Benedetto Fiorelli 346 RIPE NCC 347 P.O. Box 10096 348 Amsterdam 1001EB 349 The Netherlands 351 Phone: +31.20.535.4444 352 Email: bfiorell@ripe.net