idnits 2.17.1 draft-daigle-ura-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-19) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 918 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 155 instances of too long lines in the document, the longest one being 15 characters in excess of 72. ** There are 33 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 70 has weird spacing: '...ibility of a ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'SILKURA' is defined on line 393, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'IIAW95' -- Possible downref: Non-RFC (?) normative reference: ref. 'JAVA' ** Obsolete normative reference: RFC 1738 (Obsoleted by RFC 4248, RFC 4266) -- Possible downref: Non-RFC (?) normative reference: ref. 'SILK' -- Possible downref: Non-RFC (?) normative reference: ref. 'SILKURA' -- Possible downref: Non-RFC (?) normative reference: ref. 'TACOMA' -- Possible downref: Non-RFC (?) normative reference: ref. 'TCL' -- Possible downref: Non-RFC (?) normative reference: ref. 'TELE' Summary: 13 errors (**), 0 flaws (~~), 4 warnings (==), 10 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet-Draft Leslie Daigle 3 draft-daigle-ura-01.txt Peter Deutsch 4 November 22, 1995 Bill Heelan 5 Chris Alpaugh 6 Mary Maclachlan 7 Bunyip Information Systems, Inc 9 Uniform Resource Agents (URAs) 11 ---------------------------------------- 12 Abstract 13 ---------------------------------------- 15 This paper presents an experimental architecture for an agent system that 16 provides sophisticated Internet information access and management. Not a 17 generalized architecture for active objects that roam the Internet, these 18 agents are modeled as extensions of existing pieces of the Internet information 19 infrastructure. The proposed agent technology focuses on the necessary 20 information structures to encapsulate Internet activities into objects that can 21 be activated, transformed, and combined into larger structured activities. 23 ---------------------------------------- 24 Status of this Memo 25 ---------------------------------------- 27 This document is an Internet-Draft. Internet-Drafts are working 28 documents of the Internet Engineering Task Force (IETF), its areas, 29 and its working groups. Note that other groups may also distribute 30 working documents as Internet-Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six 33 months. Internet-Drafts may be updated, replaced, or obsoleted by 34 other documents at any time. It is not appropriate to use 35 Internet-Drafts as reference material or to cite them other than 36 as a "working draft" or "work in progrss." 38 To learn the current status of any Internet-Draft, please check the 39 1id-abstracts.txt listing contained in the Internet-Drafts Shadow 40 Directories on ds.internic.net, nic.nordu.net, venera.isi.edu, or 41 munnari.oz.au. 43 ---------------------------------------- 44 Acknowledgements 45 ---------------------------------------- 47 Several people have shared throughts and viewpoints that have helped shape 48 the thinking behind this work over the past few years. We'd like to thank, 49 in particular, Chris Weider, Patrik Faltstrom, Michael Mealling, Alan 50 Emtage, and the participants in the IETF URI Working Group for many 51 thought-provoking discussions. 53 ---------------------------------------- 54 Introduction 55 ---------------------------------------- 57 The evolution of Internet information systems has been characterized by 58 building upon successive layers of encapsulated technologies. Machine address 59 numbers were devised, and then encapsulated in advertised machine names, which 60 has allowed the evolution of the Domain Name System (DNS) [RFC1034, RFC1035]. 61 Protocols were developed for accessing Internet resources of various 62 descriptions, and then uniform mechanisms for specifying resource locations, 63 standardized across protocol types, were developed (URLs) [RFC1738]. 64 Each layer of Internet information primitives has served as the building blocks 65 for the next level of abstraction and sophistication of information access, 66 location, discovery and management. 68 The work described in this paper is an experimental system designed to take 69 another step in encapsulation. While TCP/IP protocols for routing, addressing, 70 etc, have permitted the connection and accessibility of a plethora of 71 information services on the Internet, these must yet be considered a diverse 72 collection of heterogeneous resources. The World Wide Web effort is the most 73 successful to date in attempting to knit these resources into a cohesive whole. 74 However, the activity best-supported by this structure is (human) browsing of 75 these resources as documents. The Uniform Resource Agent (URA) initiative 76 explores the possibility of specifying an activity with the same kind of 77 precision accorded to resource naming and identification. 79 A fully-instantiated URA carries out a task delegated by an invoker (human or 80 otherwise). The nature of the task is determined by the agent that the invoker 81 instantiates; that originating object encapsulates some knowledge of existence 82 of relevant Internet resources and information required in order to access them. 83 In this way, URAs can be used to allow invokers to carry out high-level 84 Internet resource activities while insulating them from the details of Internet 85 protocols, etc. Also, by formally specifying a high-level Internet activity in 86 an agent, the activity can be repeated by the same or another invoker at a 87 later date, the activity can be incorporated into a still higher-level 88 activity, the agent object can be modified to carry out a related task, etc. 90 More detail describing the underlying philosophy of this particular approach 91 can be found in [IIAW95]. 93 ---------------------------------------- 94 Motivation 95 ---------------------------------------- 97 As a very simple example, consider the client task of subscribing to a 98 mailing list. There are many mechanisms for providing the user 99 information necessary to complete a subscription. Currently, all 100 applications which provide the ability to subscribe to mailing lists 101 must contain net-aware code to carry out the task once the requisite personal 102 data has been solicited from the user. Furthermore, any application program 103 that embeds the ability to subscribe in its code necessarily limits the 104 set of mailing lists to which a client can subscribe (i.e, to those types 105 foreseen by the software's creators). If, instead, there is an agent to which 106 this task can be delegated, all applications can make use of the agent, 107 and that agent becomes responsible for carrying out the necessary interactions 108 to complete the subscription. Furthermore, that agent may be a client 109 to other agents which can supply particular information about how to 110 subscribe to new types of mail servers, etc. 112 ---------------------------------------- 113 Relationship to Other Internet Agents 114 ---------------------------------------- 116 A number of Internet-aware agent and transportable code systems have been 117 released recently -- Java [JAVA], TCL [TCL] and Safe-TCL, Telescript [TELE], 118 and the TACOMA system [TACOMA], to name a few of them. Some of these systems, 119 like Java, focus on providing mechanisms for creating and distributing 120 (inter)active documents in the World Wide Web. Others, like TACOMA, have more 121 general intentions of providing environments for mobile, interacting processes. 123 While each of these systems makes its individual contribution to solving the 124 transportation, communication, and security issues normally associated with 125 agent systems, they yield more objects that exist within the Internet 126 information space. That is, while they may permit individual users to 127 have a more sophisticated interaction with a particular information resource, 128 they do not address the more general Internet problems of naming, identifying, 129 locating resources, and locating the same or similar resources again at 130 a later date. It is this set of problems that URAs specifically set out to 131 address. 133 ---------------------------------------- 134 The Experimental Architecture 135 ---------------------------------------- 137 At the centre of the URA architecture is the concept of a (persistent) 138 specification of an activity. For purposes that should become clear 139 as the expected usage of URAs is described in more detail, we choose to support 140 this concept with the following requirements of the architecture: 142 . there is a formalized environment in which these specifications 143 are manipulated (examined, executed, etc). This is referred to as a 144 URAgency. 146 . the activity specifications are modular, and independent of a 147 given URAgency environment. Thus, they exist as object constructs 148 that can be shared amongst URAgencies. There is a standardized 149 _virtual_ structure of these URA objects, although different 150 types may exist, with different underlying implementations. 152 Basic URAgency Requirements 153 --------------------------- 155 A URAgency is a software system that manipulates URA objects. In the 156 terminology of objects, a URAgency identifies the types of URAs it handles, 157 and is responsible for applying methods to objects of those types. For the 158 purposes of this experimental work, the only methods it is required to support 159 are those to get information about a given URA, and to execute a URA. 161 The expected result of applying the "get information" method to a URA 162 is a description of some or all of the URA following the standardized 163 virtual structure of a URA object, outlined below. 165 The appropriate way to "execute" a URA is to supply information for the 166 individual URA data segments (in effect, to permit the creation of an instance 167 of a virtual object), or to identify a URA instance. Again, the information 168 is to be supplied in accordance with the virtual structure below. 170 When a URAgency claims to handle a particular type of URA, it must have 171 the ability to map that URA type's implementation structure into and out 172 of the virtual, standard URA structure, it must know how to activate the URA, 173 and it must satisfy any runtime dependencies for that type of URA. 175 For example, a URA type may consist of a Pascal program binary which, when 176 run with particular command line arguments, yields information in the 177 standard URA object structure. Activating this type of URA might consist of 178 executing the Pascal binary with an input file containing all the necessary 179 data segments. A URAgency claiming to handle this sort of URA type must 180 first be able to provide an environment to execute the Pascal binary (for 181 whatever platform it was compiled), and also be able to interact with the Pascal 182 binary according to these conventions to get information about the URA, or 183 execute it. 185 As an alternative example, a URA type may consist of a script in some 186 interpreted language, with the URA object structure embedded as data structures 187 within the script. A URAgency handling this type of URA might have to be 188 able to parse the script to pull out the standard URA object structure, and 189 provide the script language interpreter for the purposes of executing the 190 URA. 192 URA Object Structure 193 -------------------- 195 In order to capture the necessary information for carrying out the type 196 of Internet activity described in the introductory paragraphs of this 197 document, six basic (virtual) components of a URA object have been identified. 198 Any implementation of a URA type is expected to be able to conform to 199 this structure within the context of a URAgency. 201 The six basic components of a URA object are: 203 URA HEADER: 204 Identification of the URA object, including a URA name, type and 205 abstract, creator name, resources required by the URA, etc. 207 ACTIVATION DATA: 208 Specification of the data elements required to carry out the URA 209 activity. For example, in the case of an Internet search for 210 "people", this could include specification of fields for person 211 name, organization, e-mail address, etc. 213 TARGETS: 214 Specification of the URL/URN's to be accessed to carry out the 215 activity. Note that, until URN's are in common use, the ability 216 to tweak URL's will be necessary. A key issue for URAs is the 217 ability to transport them and activate them far from the creator's 218 originating site. This may have implications in terms of 219 accessibility of resource sites. For example, a software search 220 created in Canada will likely access a Canadian Archie server, and 221 North American ftp sites. However, an invoker in Australia should 222 not be obliged to edit the URA object in order to render it 223 relevant in Australia. The creator, then, can use this section to 224 specify the expected type of service, with variables for the parts 225 that can be modified in context (e.g., the host name for an Archie 226 server, or a mirror ftp site). 228 EXPERIENCE INFORMATION: 229 Specification of data elements that are not strictly involved in 230 conversing with the targets in order to carry out the agent's 231 activity. This space can be used to store information from one 232 invocation of a URA instance to the next. This kind of information 233 could include date of last execution, or URLs of resources located 234 on a previous invocation of the agent. 236 ACTIVITY: 237 If URAs were strictly data objects, specifying required data and 238 URL/URN's would suffice to capture the essence of the composite 239 net interaction. However, the variability of Internet resource 240 accesses and the scope of what URAs could accomplish in the net 241 environment seem to suggest the need to give the creator some 242 means of organizing the instantiation of the component URL/URN's. 243 Thus, the body of the URA should contain a scripting mechanism 244 that minimally allows conditional instantiation of individual 245 URL/URN's. These conditions could be based on which (content) 246 data elements the user provided, or accessibility of one URL/URN, 247 etc. It also provides a mechanism for suggesting scheduling of 248 URL/URN instantiation. 250 The activity is specified by a script or program in a language specified 251 by the URA type, or by the URA header information. All the required 252 activation data, targets, and experience information are referenced 253 by their specification names. 255 RESPONSE FILTER: 256 The main purpose of the ACTIVITY module is to specify the steps 257 necessary to take the ACTIVATION DATA, contact the TARGETS, and 258 collect responses from those services. The purpose of the RESPONSE 259 FILTER module is to transform those responses into the result of 260 the URA invocation. This transformation may be along the lines of 261 reformatting some text, or it may be a more elaborate interpretation 262 (e.g., providing a relevance rating for a retrieved HTML page). 264 The response filter is specified by a script or program in a language 265 specified by the URA type, or by the URA header information. All the 266 required activation data, targets, and experience information are 267 referenced by their specification names. 269 See Appendix 1 for a more detailed description of the components of a URA. 270 Appendix 2 contains a sample virtual URA structure. 272 The Architecture in Action 273 -------------------------- 275 Having introduced the required capabilities of the URAgency and virtual 276 structure of URA objects, it is now time to elaborate on the tasks and 277 interactions that are best supported by URAs. 279 URAs are constructed by identifying net-based resources of interest (targets) 280 to carry out a particular task. The activation data component of a URA is the 281 author's mechanism for specifying (to the invoker) the elements of information 282 that are required for successful execution . An invoker creates an instance 283 of a URA object by providing data that is consistent with, or fills in, this 284 template. Such an instance encapsulates everything that the agent "needs to 285 know" in order to contact the specified target(s), make a requrest of the 286 resource ("get", or "search", etc) and return a result to the invoker. This 287 encapsulation is a sophisticated identification of the task results. 289 For exmaple, in the case of a mailing list subscription URA, the creator will 290 identify the target URL for a resource that handles list subscription (e.g., an 291 HTML form), and specify the data required by that resource (user name, user 292 mail address, mailing list identifier, etc). When an invoker provides that 293 information and instantiates the URA, the resulting object completely 294 encapsulates all that is needed in order to subscribe the user -- the 295 subscription result is identified. 297 URAs are manipulated through the application of methods. This, in turn , is 298 governed by the URAgency with which the invoker is interacting. However, 299 because the virtual structure of URAs is represented consistently across 300 URA types and URAgencies, a URAgency can act as one of the targets of a URA. 301 Since methods can be applied to URAs remotely, URAs can act as invokers of URAs. 302 This can yield a complex structure of task modules. 304 For example, a URA designed to carry out a generalized search of book-selling 305 resources might make use of individual URAs tailored to each resource. Thus, 306 the top-level URA becomes the orchastrating URA for access to a number of 307 disparate resources, while being insulated from the minute details of 308 accessing those resources. 310 ---------------------------------------- 311 A Prototype Implementation 312 ---------------------------------------- 314 The experimental work with URAs includes a prototype implementation of URA 315 objects. These are written in the Tcl scripting language. A sample prototype 316 Tcl URA can be found in Appendix 3. 318 The URAgency that was created to handle these URAs is part of the Silk 319 Desktop Internet Resource Discovery tool. Silk provides a graphical user 320 interface environment that allows the user to access and search for 321 Internet information without having to know where to look or how to look. 322 Silk presents a list of the available URAs to carry out these activities 323 (e.g., "search for tech reports", "hotlist", etc). For each activity, the 324 user is prompted for the activation data, and Silk's URAgency executes the 325 URA. The Silk software also supports the creation and maintenance of URA 326 object instances. Users can add new URAs by creating new Tcl scripts (per 327 the guidelines in the "URA Writer's Guide", available with the Silk software. 328 See [SILK]). The Silk graphical interface hides some of the mechanics of the 329 underlying URAgency. A more directly-accessible version of this URAgency 330 will become available. 332 ---------------------------------------- 333 Conclusions 334 ---------------------------------------- 336 This work was originally conceived as an extension to the family of 337 Uniform Resource Identifiers -- URLs, and the proposed Uniform Resource 338 Names (URNs), and Uniform Resource Characteristics (URCs). The approach 339 of formalizing the characteristics of an information task in a standardized 340 object structure is seen as a means of identifying a class of resources, 341 and contributes to the level of abstraction with which users can refer 342 to Internet resources. 344 Although still in experimental stages, this work has already evoked interest 345 and shown promise in the area of providing mechanisms for building more 346 advanced tools to interact with the Internet at a more sophisticated level than 347 just browsing web pages. 349 One of the major difficulties that has been faced in developing a collection 350 of URAs is the brittleness induced by interacting with services that are 351 primarily geared towards human-users. Small changes in output formats 352 (easily discernable by the human eye) can be entirely disruptive to a 353 software client that must apply a parsing and interpretation mechanism 354 based on placement of cues in the text. This problem is certainly not 355 unique to URAs -- any software acting upon results from such a service 356 is affected. Perhaps there is the need for an evolution of "service entrances" 357 to information servers on the Internet -- mechanisms for getting "just 358 the facts" from an information server. Of course, one way to provide 359 such access is for the service provider to develop and distribute a URA 360 that interacts with the service. When the service's interface changes, 361 the service provider will be moved to update the URA that was built to 362 access it reliably. 364 Work will continue to develop new types of URAs, as well as other URAgencies. 365 This will necessitate the creation of URAgency interaction standards -- 366 the "common virtual URA object structure" is the first step towards defining 367 a lingua franca among URAs of disparate types and intention. 369 ---------------------------------------- 370 References 371 ---------------------------------------- 373 [IIAW95] Leslie L. Daigle, Peter Deutsch, "Agents for Internet Information 374 Clients", CIKM'95 Intelligent Information Agents Workshop, December 1995. 375 Available from 376 378 [JAVA] "The Java Language: A White Paper" Available from 379 381 [RFC1034] P.V. Mockapetris, "Domain Names - Concepts and Facilities", 382 RFC 1034, November 1987. 384 [RFC1035] P.V. Mockapetris, "Domain Names - Implementation and 385 Specification", RFC 1035, November 1987. 387 [RFC1738] T. Berners-Lee, L. Masinter, M. McCahill, "Uniform Resource 388 Locators (URL)", RFC 1738, December 1994. 390 [SILK] Bunyip's Silk project homepage: 391 393 [SILKURA] Silk URA information: 394 396 [TACOMA] Johansen, D. van Renesse, R. Schneider, F. B., "An Introduction to 397 the TACOMA Distributed System", Technical Report 95-23, Department of 398 Computer Science, University of Tromso, Norway, June 1995. 400 [TCL] Ousterhout, J. K. "Tcl and the Tk Toolkit", Addison Wesley, 1994. 402 [TELE] White, J. E., "Telescript Technology: The Foundation for the 403 Electronic Marketplace", General Magic White Paper, General Magic Inc., 404 1994. 406 ---------------------------------------- 407 Authors' Address 408 ---------------------------------------- 410 Leslie Daigle 411 Peter Deutsch 412 Bill Heelan 413 Chris Alpaugh 414 Mary Maclachlan 416 Bunyip Information Systems, Inc. 417 310 St. Catherine St. West 418 Suite 300 419 Montreal, Quebec, CANADA 420 H2X 2A1 422 Phone: (514) 875-8611 424 E-mail: ura-bunyip@bunyip.com 426 ---------------------------------------- 427 Appendix 1 -- Virtual URA Structure 428 ---------------------------------------- 430 This appendix contains a BNF-style description of the expected virtual 431 structure of a URA object. This "virtual structure" acts as the canonical 432 representation of the information encapsulated in a given URA. It is expected 433 that more information may optionally be contained in the elements of the 434 components -- the elements listed here are offered as the "minimum" or 435 "standard" set. 437 N.B.: 438 []-delimited items are optional 439 %% denotes a comment 440 \0 represents the empty string 441 | is "or" 442 {} are literal characters 444 This form is used for convenience and clarity of expression -- whitespace and 445 ordering of individual elements are not considered significant. 447 := {} 449 := { URAHDR } 450 { ACTDATA } 451 { TARG } 452 { EXPINFO } 453 { ACTSPEC } 454 { RESPFILT } 456 := { name } 457 { author } 458 { version } 459 [ { lang } ] 460 [ { parent } ] 462 := | \0 464 := { 465 { name } 466 { response } 467 { prompt } 468 [ { required } ] 469 [ { default } ] 470 } 472 := | \0 474 := { 475 { name } 476 { protocol } 477 { url } 478 [ { } ] 479 } 481 := | 483 := %% a complete, valid URL string (e.g., http://www.bunyip.com/) 485 := { 486 { scheme } 487 { host } 488 [ { port } ] 489 { selector } 490 } 492 := { 493 { name } 494 { response } 495 { prompt } 496 } 497 := { 498 { name } 499 { response } 500 { prompt } 501 } 502 := { 503 { name } 504 { response } 505 { prompt } 506 } 507 := { 508 { name } 509 { response } 510 { prompt } 511 } 513 := { 514 { name } 515 { response } 516 } 518 := 520 := 522 %% Without requiring more detail... 524 := \n | \0 525 := 0 | 1 526 := 527 := 528 := 529 := 530 := 531 := 532 := 533 := 534 := 535 := 536 := 537 := http-get | http-post | ... 538 := 539 := 540 := 541 := 542 := 543 := 544 := 545 := 546 := 547 := 548 := 549 := 550 := 552 ---------------------------------------- 553 Appendix 2 -- Sample Virtual URA 554 Representation 555 ---------------------------------------- 557 A valid virtual representation of a Silk Tcl URA is presented below. The 558 actual URA from which it was drawn is given in Appendix 3. 560 { 561 {URAHDR 562 {name {DejaNews Search}} 563 {author {Leslie Daigle}} 564 {version {1.0}} 565 } 567 {ACTDATA 568 {name {Topic Keywords}} 569 {prompt {Topic Keywords}} 570 {response {}} 571 } 573 {EXPINFO 574 {name {Comments}} 575 {prompt {Comments}} 576 {response {}} 577 } 579 {ACTSPEC 580 {proc mapResponsesToDejanews {} { 581 set resp "" 582 if {[uraAreResponsesSet {Topic Keywords}]} { 583 lappend resp [list query [uraGetSpecResponse {Topic Keywords}]] 584 } 586 return $resp 588 } 589 proc uraRun {} { 590 global errorInfo 592 foreach serv [uraListOfServices] { 593 set u [uraGetServiceURL $serv] 595 switch -- $serv { 596 dejanews { 597 if [catch { 598 set query [mapResponsesToDejanews] 599 if {$query != {}} { 600 set result [uraHTTPPostSearch $u $query] 601 if {$result != ""} { 602 set list [dejanews_uraHTTPPostCanonicalize $result] 603 puts $list 604 } 605 } 606 }] { 607 puts stderr $errorInfo 608 } 609 } 611 default { 612 # can't handle other searches, yet. 613 } } } } 614 } 615 } 617 {RESPFILT 618 { 619 proc dejanews_uraHTTPPostCanonicalize {htmlRes} { 621 set result {} 622 set lines {} 623 set clause {} 624 set garb1 "" 625 set garb2 "" 627 # Get the body of the result page -- throw away leading and 628 # trailing URLs 630 regexp {([^
]*)
(.*)
.*} $htmlRes garb1 garb2 mainres

632	         set lines [split $mainres "\n"]

634	         foreach clause $lines {

636	           if [regexp {
.*(..\/..).*([^<]*).*([^<]*).*} \
637	                $clause garb1 dt relurl desc grp] {

639	             lappend r [list HEADLINE [format "%s    (%s, %s)" [string trim $desc] \
640	                 [string trim  $grp] $dt]]
641	             lappend r [list URL [format "http://www.dejanews.com/cgi-bin/%s" $relurl]]
642	             lappend r [list TYPE "text/plain"]

644	             lappend result $r
645	           }
646	         }
647	         return $result
648	       }
649	      }

651	   }

653	}

655	----------------------------------------
656	Appendix 3  -- Sample Silk Tcl URA
657	----------------------------------------

659	The following is a valid Silk Tcl URA.  For more information on the
660	implementation and structure of Silk-specific URAs, see the "URA Writers
661	Guide" that accompanies the distribution of the Silk software (available
662	from ).

664	# -----------------------------------------------------------------------------
665	#
666	#                             URA initialization
667	#
668	# -----------------------------------------------------------------------------

670	#
671	# Initialize the URA, its search specs and searchable services.
672	#

674	# URA init.

676	set uraDebug 1

678	uraInit {
679	  {name {DejaNews Search}}
680	  {author {Leslie Daigle}}
681	  {version {1.0}}
682	  {description "This URA will search for UseNet News articles."}
683	  {help "This is help on UseNet News search script."}
684	}

686	#
687	# bug: handling of choices/labels is kind of gross.
688	#

690	# Search spec. init.

692	foreach item {
693	  {
694	    {name        {Topic Keywords}}
695	    {field       Topic}
696	    {tag         STRING}
697	    {description {Keywords to search for in news articles}}
698	    {prompt      {Topic Keywords}}
699	    {help        {Symbols to look up, separated by spaces.}}
700	    {type        STRING}
701	    {subtype     {}}
702	    {allowed     .*}
703	    {numvals     1}
704	    {required    0}
705	    {response    {}}
706	    {respset     0}
707	  }
708	} {
709	  uraSearchSpecInit $item
710	}

712	uraAnnotationInit {
713	  {help        {Enter comments to store with an instance}}
714	  {numvals     1}
715	  {subtype     {}}
716	  {response    {}}
717	  {name        Comments}
718	  {required    0}
719	  {class       ANNOTATION}
720	  {type        TEXT}
721	  {description {General comments about this URA.}}
722	  {respset     1}
723	  {prompt      Comments}
724	  {field       {}}
725	  {allowed     .*}
726	}

728	uraResultInit {
729	  {name {Related Pages}}
730	  {contents { {
731	    {HEADLINE {The DejaNews UseNet search service}}
732	    {TYPE text/plain}
733	    {URL http://www.dejanews.com}
734	} }}
735	}

737	foreach item {
738	  {
739	    {name dejanews}
740	    {protocol http-post}
741	    {url http://marge.dejanews.com/cgi-bin/nph-dnquery}
742	  }
743	} {
744	  uraServicesInit $item
745	}

747	proc dejanews_uraHTTPPostCanonicalize {htmlRes} {

749	  set result {}
750	  set lines {}
751	  set clause {}
752	  set garb1 ""
753	  set garb2 ""

755	  # Get the body of the result page -- throw away leading and trailing URLs
756	  regexp {([^
]*)
(.*)
.*} $htmlRes garb1 garb2 mainres

758	  set lines [split $mainres "\n"]

760	  foreach clause $lines {

762	    uraDebugPuts stderr [format "Line: %s" $clause]

764	    if [regexp {
.*(..\/..).*([^<]*).*([^<]*).*} \
765	         $clause garb1 dt relurl desc grp] {
766	      uraDebugPuts stderr [format "Date: %s Rel URL: %s Desc: %s Group: %s" \
767	             $dt $relurl $desc $grp]

769	      lappend r [list HEADLINE [format "%s    (%s, %s)" [string trim $desc] \
770	          [string trim  $grp] $dt]]
771	      lappend r [list URL [format "http://www.dejanews.com/cgi-bin/%s" $relurl]]
772	      lappend r [list TYPE "text/plain"]

774	      lappend result $r
775	    }
776	  }
777	  return $result

779	}

781	# -----------------------------------------------------------------------------
782	#
783	#                             Mapping procedures
784	#
785	# -----------------------------------------------------------------------------

787	#
788	# There is one procedure, for each searchable service, to map the search
789	# spec responses to a form suitable for inclusion into a search URL (or
790	# whatever form the particular query procedure accepts).
791	#

793	#
794	#
795	proc mapResponsesToDejanews {} {
796	  set resp ""
797	  if {[uraAreResponsesSet {Topic Keywords}]} {
798	    lappend resp [list query [uraGetSpecResponse {Topic Keywords}]]
799	  }

801	  return $resp

803	}

805	#
806	# bug: need better error reporting (i.e. which searches didn't work and why, etc.)
807	#
808	proc uraRun {} {
809	  global errorInfo

811	  foreach serv [uraListOfServices] {
812	    set u [uraGetServiceURL $serv]

814	    switch -- $serv {
815	      dejanews {
816	        if [catch {
817	          set query [mapResponsesToDejanews]
818	          uraDebugPuts stderr [format "%s: query is `%s'." $serv $query]
819	          if {$query != {}} {
820	              set result [uraHTTPPostSearch $u $query]
821	              if {$result != ""} {
822	                uraDebugPuts stderr [format "%s: result is `%s'." $serv $result]
823	                set list [dejanews_uraHTTPPostCanonicalize $result]
824	                uraDebugPuts stderr [format "%s: list is `%s'." $serv $list]
825	                puts $list
826	              }
827	            }
828	        }] {
829	          puts stderr $errorInfo
830	        }
831	      }

833	      default {
834	        # can't handle other searches, yet.
835	      }
836	    }
837	  }
838	}

840	------------------------------------------------------------------------------

842	"The Relentless Pursuit of Perfection                   Leslie Daigle
843	   is a depth-first search of the infinitely deep       leslie@bunyip.com
844	      tree of life's experiences."                      Montreal, Canada
845	                                        -- ThinkingCat

847	------------------------------------------------------------------------------