idnits 2.17.1 draft-vanwijk-sipping-deaf-req-00.txt: ** The Abstract section seems to be numbered -(223): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(798): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** Missing revision: the document name given in the document, 'draft-vanwijk-sipping-deaf-req-', does not give the document revision number ~~ Missing draftname component: the document name given in the document, 'draft-vanwijk-sipping-deaf-req-', does not seem to contain all the document name components required ('draft' prefix, document source, document name, and revision) -- see https://www.ietf.org/id-info/guidelines#naming for more information. == Mismatching filename: the document gives the document name as 'draft-vanwijk-sipping-deaf-req-', but the file name used is 'draft-vanwijk-sipping-deaf-req-00' ** The document is more than 15 pages and seems to lack a Table of Contents. == There are 4 instances of lines with non-ascii characters in the document. == The page length should not exceed 58 lines per page, but there was 20 longer pages, the longest (page 2) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 385 instances of too long lines in the document, the longest one being 5 characters in excess of 72. ** The abstract seems to contain references ([3], [6], [7], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 691: '...al manufacturers SHOULD adhere to. Kee...' RFC 2119 keyword, line 698: '...(from now on just named terminal) MUST...' RFC 2119 keyword, line 701: '...perator who to call. It SHOULD be done...' RFC 2119 keyword, line 708: '...roviders and terminals MUST be able to...' RFC 2119 keyword, line 714: '...hat modern terminals SHOULD be able to...' (36 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Couldn't figure out when the document was first submitted -- there may comments or warnings related to the use of a disclaimer for pre-RFC5378 work that could not be issued because of this. Please check the Legal Provisions document at https://trustee.ietf.org/license-info to determine if you need the pre-RFC5378 disclaimer. -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Missing reference section? '10' on line 982 looks like a reference -- Missing reference section? '6' on line 963 looks like a reference -- Missing reference section? '7' on line 969 looks like a reference -- Missing reference section? '1' on line 944 looks like a reference -- Missing reference section? '3' on line 952 looks like a reference -- Missing reference section? '8' on line 974 looks like a reference -- Missing reference section? '9' on line 978 looks like a reference -- Missing reference section? '2' on line 948 looks like a reference -- Missing reference section? '11' on line 985 looks like a reference -- Missing reference section? '12' on line 989 looks like a reference -- Missing reference section? '13' on line 994 looks like a reference -- Missing reference section? '14' on line 997 looks like a reference -- Missing reference section? '15' on line 1000 looks like a reference -- Missing reference section? '5' on line 960 looks like a reference -- Missing reference section? '4' on line 956 looks like a reference -- Missing reference section? '16' on line 1005 looks like a reference -- Missing reference section? '17' on line 1018 looks like a reference Summary: 10 errors (**), 1 flaw (~~), 5 warnings (==), 18 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force SIPPING WG 3 Internet Draft van Wijk/Rosenberg/ 4 Document: Schulzrinne 6 July 2001 Ericsson/Dynamicsoft 7 Expires: December 2001 /WCOM/Columbia U. 8 Category: Informational 10 SIP Support for Hearing and Speech Impaired Users 12 Status of this Memo 14 This document is an Internet-Draft and is in full conformance with 15 all provisions of Section 10 of RFC2026 [10]. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. Internet-Drafts are draft documents valid for a maximum of 20 six months and may be updated, replaced, or obsoleted by other 21 documents at any time. It is inappropriate to use Internet-Drafts 22 as reference material or to cite them other than as "work in 23 progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html. 30 1. Abstract 32 SIP is attracting more and more attention as a valuable tool to 33 enable and support voice and multimedia communications over the 34 Internet. However, some users of SIP-based services will be unable 35 or severely restricted to use plain voice communication. In 36 particular, people who are hearing impaired often cannot send 37 and/or receive voice. This document is a merger between 2 38 previously written drafts [6] and [7] regarding SIP and The 39 Hearing Impaired and its goal is to lay a foundation for design 40 and implementation of SIP based services. 41 These services are generally enabled by baseline SIP [1], or 42 through the use of the caller preferences specification [3]. No 43 additional extensions are proposed here in order to support 44 universal access. 46 2. Terminology and Conventions Used in This Document 48 In this document, the key words "MUST", "MUST NOT", 49 "REQUIRED","SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", 50 "RECOMMENDED","MAY", and "OPTIONAL" are to be interpreted as 51 described in RFC2119 [8] and indicate requirement levels for 52 compliant SIP implementations. For the purposes of this document, 54 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 1 of 21] 55 the term terminal will be used to represent the hearing impaired 56 individual and his/her SIP-enabled end-user equipment; this could 57 be a TTY[9], a personal computer or PDA, a specially equipped 58 mobile phone, video phone, etc. The term user, in this document, 59 shall be understood to mean a hearing or speech impaired 60 individual. 62 3. Introduction 64 The Session Initiation Protocol (SIP)[1] is used to initiate, 65 modify, and terminate interactive sessions between sets of users. 66 Typically these are voice sessions, described by the Session 67 Description Protocol (SDP)[2]. However not all users or potential 68 users have access to SIP services. Specifically, people who are 69 Culturally Deaf, Audiologically deaf, hard of hearing, speech 70 impaired or disabled, etc., on either a temporary or permanent 71 basis, are unable to participate in plain voice-based 72 communications. For the purposes of this document, this group of 73 people will be referred to collectively as Hearing Impaired 74 individuals. Also, the term Sign Language (SL), is used in this 75 document to refer to the natural signed language used by the Deaf. 76 In the United States and parts of Canada and Mexico, American Sign 77 Language (ASL) is used. For Deaf individuals in other countries, 78 for example Britain, British Sign Language (BSL) is used. Also, 79 since Deaf people can and do use various types of manual 80 communications systems in addition to SL such as Signed English 81 for this document the term SL will be assumed to cover all forms 82 of manual communications. 83 Within the Public Switched Telephone System (PSTN), services have 84 been defined that allow for access to circuit switched based relay 85 voice services by these users. We believe it is important to offer 86 similar or better services in an IP context. The flexibility of 87 SIP affords us the ability to both offer and improve on these 88 services, and to offer more extensive forms of universal service 89 access to this group of customers. This is a formative time for 90 the future of IP-based communications and as such it is an 91 appropriate time to ensure that such requirements as are necessary 92 to ensure full accessibility by all customers are included in 93 planning the new networks using SIP, such as the 3G wireless 94 network. 96 4. Purpose and Scope 98 This document will first describe a few possible services using 99 call flows that enable universal access of voice and multimedia 100 sessions, initiated by SIP to users who are hearing impaired. And 101 second, offers a proposed set of requirements based on the 102 services and attempts to identify issues that need resolution to 103 allow full accessibility for all customers to SIP-based Internet 104 communications. 105 These services are generally enabled by baseline SIP[1], or 106 through the use of the caller preferences specification [3]. No 108 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 2 of 21] 109 additional extensions are proposed here in order to support 110 universal access. 112 5. Background 114 In the current telephony world, most hearing impaired individuals 115 have access to either standalone or PC-based TTY devices. These 116 text-based devices or software packages are adequate for 117 communication with other hearing impaired individuals or with 118 hearing individuals that have similar devices or software. In 119 addition, most if not all states in the United States have state- 120 sponsored relay service, in which a human operator with both 121 standard telephone equipment and TTY devices acts as a go-between 122 ("relay operator" or interpreter), allowing a hearing impaired 123 person with a TTY to place or receive calls from hearing 124 individuals with standard telephones. Individuals with speech 125 difficulties that render it impossible for the individual to use 126 ordinary voice-enabled devices can use the same TTY equipment and 127 relay services. Introduction of the relay service has been a great 128 benefit to both the hearing impaired and hearing people because it 129 has enabled communication between them, where, prior to this time, 130 it was difficult or impossible. It has provided independence to 131 hearing impaired people, allowing them to communicate on their own 132 terms rather than being forced to rely on hearing friends to act 133 as telephone interpreters. 135 In recent years, other technology such as two-way e-mail pagers 136 have become available that provide portable communications for 137 hearing impaired people in a manner similar to cellular telephones 138 for hearing people. These have provided a great benefit to hearing 139 impaired people, allowing them the ease of near-instant 140 communications that hearing people now take for granted. Some e- 141 mail pager services have also supported interface with other 142 devices such as TTYs and FAX. In addition, most European GSM 143 phones and other mobile services elsewhere offer SMS (short 144 message service) for exchanging short text messages to other 145 subscribers. This will also allow hearing impaired people to 146 communicate near-instantly. 148 Most recently, the advent of video relay services has provided 149 ways for hearing impaired individuals to converse with hearing 150 individuals using their most natural means of communication, 151 visual, through the use of an oral or sign language interpreter. 152 The human interpreter is typically physically located at the relay 153 center and provides interpreting services via video connection to 154 the hearing impaired person and voice connection to the hearing 155 person. 157 It is not easy for a hearing person to understand the strong need 158 for the hearing impaired people to have a reliable and easy to use 159 relay service. But imagine yourself to live one day without a 160 telephone. From doing business to simply ordering a pizza, the 161 telephone is often unavoidable today. And that is no small feat. 163 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 3 of 21] 164 Imagine then how this is if this was not just one day, but for the 165 rest of your life? Now you may get an idea why the relay service 166 is so important for the hearing impaired people. Even a small 167 improvement will be received as a mayor improvement. 169 The reader of this draft is supposed to have at least a 170 superficial knowledge and understanding of the deaf and hard of 171 hearing world and the communication problems that have been 172 endemic in that population since the creation of voice-only, long- 173 distance communications. This may not be realistic, so the authors 174 recommend the interested readers the following books as 175 supplemental readings [11] [12] [13] [14]. 177 6. Example Services and Call Flows 179 We provide the following examples services and accompanying call 180 flows: 182 1 - Redirect to IM: 183 The caller has a phone and an IM client. The called party has a 184 phone and an IM client. The phone call is redirected to IM and 185 both parties use IM to communicate. 187 2 - One-way speech to text translation service: 188 The caller has only a phone. The called party has a text terminal 189 to receive and a phone to send. A relay service translates in one 190 direction only from speech to text. 192 3 - One-way speech to sign language translation service: 193 The caller has just a phone. The called party has a video terminal 194 to receive and a phone to send. A relay service translates in one 195 direction only from speech to video, with the video being a sign 196 language representation of the speech. 198 4 - Two-way speech to text and text to speech with translation 199 service: 200 The caller has a phone only. The called party uses text both ways. 201 A relay service translates in one direction from text to speech 202 and from speech to text in the other direction. A computer can do 203 the text to speech translation. 205 5 - Hearing impaired calling party calling through relay: 206 The caller has text only. The called party only has a phone. A 207 relay service translates in one direction from text to speech and 208 from speech to text in the other direction. A computer can do the 209 text to speech translation. 211 6.1 Redirect to IM 213 One advantage of providing SIP based voice services through the 214 Internet is the seamless access to other IP services that can be 215 used in conjunction with voice. Instant Messaging (IM) is 216 particularly useful for the hearing impaired. IM allows for 218 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 4 of 21] 219 instantaneous text messaging between IP connected users. Using SIP 220 for IM service is recently described [15]. 222 One way to use IM to support the hearing impaired is to redirect a 223 voice call to an IM �call� (provided the caller supports IM). The 224 service works as follows. A voice call is initiated by a terminal 225 that can support IM as well. Indication of support for IM is done 226 through the caller preferences specification [3], which allows the 227 caller to indicate characteristics of the URLs they are willing to 228 be redirected to. In this case, they would indicate support of the 229 MESSAGE method, used for instant messaging within SIP. Support for 230 other instant messaging protocols, so long as they are described 231 by standardized URL schemes, can also be indicated. 233 When the call arrives at the user agent of the hearing impaired 234 user, the UA checks for support of instant messaging. If such 235 support is indicated, the UAS sends a 302 (Use IM - Hearing 236 Impaired) redirect, containing a URL to be used for IM. This 237 redirect is forwarded back to the calling party, whose IM tool 238 pops up with an IM filled in with the address of the called party. 239 The two can then communicate in a pure IM session. 241 The service can also be provided by an application server serving 242 the hearing impaired user. The application server, upon receiving 243 the INVITE, would initiate its own INVITE towards the hearing 244 impaired user (without indicating any kind of media session). This 245 has the effect of alerting (through a flashing light or some other 246 means) that an incoming call is taking place. If accepted, the 247 application server can then redirect the initial caller to send an 248 IM to a pre-configured IM address. 250 Figure 1 contains a call flow for the service assuming it is being 251 provided by the called UA. 253 | | 254 | F1: INVITE | 255 | --------------------------> | 256 | | 257 | | 258 | F2: 302 IM | 259 | <-------------------------- | 260 | | 261 | | 262 | F3: ACK | 263 | --------------------------> | 264 | | 265 | | 266 | | 267 | | 268 | F4: MESSAGE | 269 | --------------------------> | 270 | F5: 200 OK | 272 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 5 of 21] 273 | <-------------------------- | 274 | | 275 | F6: MESSAGE | 276 | <-------------------------- | 277 | F7: 200 OK | 278 | --------------------------> | 279 | | 281 Caller Hearing 282 Impaired 283 User 285 Figure 1: Redirecting to an IM 287 Message F1 is: 289 INVITE sip:hiu@example.com SIP/2.0 290 Via: SIP/2.0/UDP a.example.com 291 From: sip:caller@example.com 292 To: sip:hiu@example.com 293 Call-ID: 9asdg9a7@1.2.3.4 294 CSeq: 1 INVITE 295 Contact: sip:caller@a.example.com 296 Accept-Contact: *;methods=''MESSAGE,SUBSCRIBE'' 297 Content-Type: application/sdp 298 Content-Length: XX 300 302 Message F2 is: 304 SIP/2.0 302 Use IM - Hearing Impaired 305 Via: SIP/2.0/UDP a.example.com 306 From: sip:caller@example.com 307 To: sip:hiu@example.com;tag=9ajsd9aumlaa 308 Call-ID: 9asdg9a7@1.2.3.4 309 CSeq: 1 INVITE 310 Contact: sip:hiu@example.com;method=MESSAGE 312 6.2 One-way Speech-to-text Translation Service 314 The above IM service is very interesting and quite useful for 315 short messages. But when a hearing impaired user needs to place or 316 receive a telephone call, a relay service is necessary. A relay is 317 a person who can listen to the calling party, type up the text, 318 and send it to the hearing impaired user either through instant 319 messages or through text over RTP [5]. 320 In one variant of this service, a call is made to a hearing 321 impaired person. If the hearing impaired user wishes to accept the 322 call, they send a 183 (Using a Relay for Hearing Impaired) 323 response to the call. 325 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 6 of 21] 326 The provisional response to the caller is used by the client to 327 alert the caller to the fact that the called party is hearing 328 disabled and that a relay service will be part of the call. This 329 is useful to help the caller to tune the speaking style, so as to 330 adjust for such a type of communication. 331 Then, after sending the 183, using the third party call control 332 mechanisms [4], the called party launches a call to a relay 333 server, with the INVITE containing SDP that indicates support for 334 only the RTP payload format for text messages. The response from 335 the relay speech to text (STT) server (presumably accepting the 336 call), contains SDP where the relay server expects to receive 337 audio to be translated to text. When this 200 OK arrives at the 338 hearing impaired user, that SDP is placed into the 200 OK of the 339 call. The result is that the caller will be sending media to the 340 relay server, and the hearing impaired user will receive a textual 341 version of it over RTP. However, the hearing impaired user sends 342 audio directly to the caller voice carry over (VCO), assuming that 343 the hearing impaired user is able to communicate verbally). When 344 this is the case, it has the advantage of sending the speech 345 directly between the participants in the direction that is 346 possible, resulting in less latency and more privacy for the 347 callers since the relay will now hear only half of the 348 conversation. With the current PSTN, the relay interpreter would 349 hear the whole conversation when VCO is enabled (relay acts then 350 as a conference bridge in VCO mode). 352 The call flow for this service is depicted in Figure 2. 354 Message F1 is: 356 INVITE sip:hiu@example.com SIP/2.0 357 Via: SIP/2.0/UDP a.example.com 358 From: sip:caller@example.com 359 To: sip:hiu@example.com 360 Call-ID: 9asdg9a7@1.2.3.4 361 CSeq: 1 INVITE 362 Contact: sip:caller@a.example.com 363 Accept-Contact: *;methods=''MESSAGE,SUBSCRIBE'' 364 Content-Type: application/sdp 365 Content-Length: XX 367 369 message F2 is: 371 SIP/2.0 183 Using Relay for Hearing Impaired... Please Wait 372 Via: SIP/2.0/UDP a.example.com 373 From: sip:caller@example.com 374 To: sip:hiu@example.com;tag=9ajsd9aumlaa 375 Call-ID: 9asdg9a7@1.2.3.4 376 CSeq: 1 INVITE 378 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 7 of 21] 379 message F3 is: 381 INVITE sip:speech2txt@example.com SIP/2.0 382 Via: SIP/2.0/UDP b.example.com 383 From: sip:hiu@example.com 384 To: sip:speech2txt@example.com 385 Call-ID: 88725392k@4.3.2.1 386 CSeq: 7 INVITE 387 Contact: sip:hiu@b.example.com 388 Content-Type: application/sdp 389 Content-Length: XX 391 393 message F4 is: 395 SIP/2.0 200 OK - translating 396 Via: SIP/2.0/UDP b.example.com 397 From: sip:hiu@example.com 398 To: sip:speech2txt@example.com;tag=1238827819 399 Call-ID: 88725392k@4.3.2.1 400 CSeq: 7 INVITE 401 Contact: sip:speech2txt@c.example.com 402 Content-Type: application/sdp 403 Content-Length: XX 405 407 message F5 is: 409 SIP/2.0 200 OK 410 Via: SIP/2.0/UDP a.example.com 411 From: sip:caller@example.com 412 To: sip:hiu@example.com;tag=9ajsd9aumlaa 413 Call-ID: 9asdg9a7@1.2.3.4 414 CSeq: 1 INVITE 415 Content-Type: application/sdp 416 Content-Length: XX 418 420 | | | 421 | F1: INVITE | | 422 | ---------------------> | | 423 | | | 424 | F2: 183 | | 425 | <--------------------- | | 426 | | F3: INVITE | 427 | | ----------------------> | 428 | | | 429 | | F4: 200 OK | 430 | | <---------------------- | 432 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 8 of 21] 433 | F5: 200 OK | | 434 | <--------------------- | | 435 | | | 436 | | | 437 | F6: ACK | | 438 | ---------------------> | | 439 | | F7: ACK | 440 | | ----------------------> | 441 | | | 442 | | | 443 | | | 444 | | | 445 | RTP (audio) | | 446 | ===============================================> | 447 | <===================== | | 448 | | | 449 | | | 450 | | RTP (text) | 451 | | <====================== | 452 | | | 453 | | | 454 | | | 455 | | | 456 | | | 457 | | | 459 Caller Hearing Relay 460 Impaired STT 461 User 463 Figure 2: One Way Translation Service 465 message F6 and F7 are standard ACK messages, not shown. 467 6.3 One-way Speech-to-Sign-Language Translation Service 469 The caller from a normal phone makes a call to a hearing impaired 470 user who needs to communicate with Sign language. The hearing 471 impaired user establishes a connection with a Relay service that 472 will listen to speech and "convert" it to sign language. The sign 473 language is sent to the hearing impaired used through a video 474 stream. 475 This service is accomplished identically to the one-way speech to 476 text translation service. The call flow is the same as listed in 477 Figure 2. The only difference is that the SDP that indicates text, 478 will instead indicate video. The RTP stream marked as containing 479 text, will instead contain video. And the Relay STT should be read 480 as Relay Speech to Sign language (STS). 482 6.4 Two-way speech to text and text to speech with Relay service 484 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 9 of 21] 485 The service in the previous section can be extended to include one 486 relay for speech to text and another that does text to speech 487 (where the text is typed by the user who is unable to communicate 488 verbally). The text to speech translation can be done by a 489 computer this is cheaper then using a human operator). If people 490 are used to translate in both directions, these translators may be 491 the same person, but they need not be. As mentioned in 6.2 this 492 has an interesting effect of introducing some form of privacy. 493 With two different translators, neither receives the complete 494 conversation, and in all likelihood, would not be able to 495 ascertain what is actually being talked about. But since Relay 496 operators adhere to privacy and security rules as mentioned in 7, 497 it is expected to be the same (human) operator. A call flow for 498 this variant on the service is shown in Figure 3. 500 Messages F1, F2, F3 and F4 are the same as above. F5 is a standard 501 ACK. 503 F6 is: 505 INVITE sip:text2speech@example.com SIP/2.0 506 Via: SIP/2.0/UDP b.example.com 507 From: sip:hiu@example.com 508 To: sip:text2speech@example.com 509 Call-ID: 87765448902@4.3.2.1 510 CSeq: 88 INVITE 511 Contact: sip:hiu@b.example.com 512 Content-Type: application/sdp 513 Content-Length: XX 515 517 and F7 looks like: 519 SIP/2.0 200 OK 520 Via: SIP/2.0/UDP b.example.com 521 From: sip:hiu@example.com 522 To: sip:text2speech@example.com;tag=9asdgnzli98a0 523 Call-ID: 87765448902@4.3.2.1 524 CSeq: 88 INVITE 525 Contact: sip:text2speech@d.example.com 526 Content-Type: application/sdp 527 Content-Length: XX 529 531 F8 is a standard ACK. F9 looks like F5 from the asymmetric version 532 of the service. 534 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 10 of 21] 535 | | | | 536 | F1: INVITE | | | 537 | ---------------------> | | | 538 | | | | 539 | F2: 183 | | | 540 | <--------------------- | | | 541 | | F3: INVITE | | 542 | | --------------------> | | 543 | | | | 544 | | F4: 200 OK | | 545 | | <-------------------- | | 546 | | | | 547 | | F5: ACK | | 548 | | --------------------> | | 549 | | | | 550 | | F6: INVITE | | 551 | | ----------------------------> | 552 | | | | 553 | | F7: 200 OK | | 554 | | <---------------------------- | 555 | | | | 556 | | F8: ACK | | 557 | | ---------------------------- >| 558 | | | | 559 | F9: 200 OK | | | 560 | <--------------------- | | | 561 | | | | 562 | | | | 563 | F10 ACK | | | 564 | ---------------------> | | | 565 | | RTP (speech) | | 566 |===============================================>| | 567 | |<======================| | 568 | | RTP (text) | | 569 | | | | 570 | | | | 571 | | RTP (text) | | 572 | |==============================>| 573 |<=======================================================| 574 | RTP (Speech) | | | 575 | | | | 576 | | | | 578 Caller Hearing STT TTS 579 Impaired 580 User 582 Figure 3: Two-way speech to text and text to speech 584 This approach has also the advantage that any application service 585 provider can be used for these translation services. Different 586 providers can be used for each direction, and these providers do 587 not need to be affiliated in any way with the ISP providing IP 589 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 11 of 21] 590 services for the hearing impaired user. This provides the 591 potential for competition, and thus improved service. 593 This approach also has the advantage of allowing one direction 594 (speech to text), the other direction (text to speech), or both, 595 to be performed by automated systems. For example, text to speech 596 technology is fairly robust, and could be used in one direction, 597 whereas a human operator could be used in the reverse (speech to 598 text) direction, since speech recognition is not (yet) that 599 robust. The call flow is completely identical, independently of 600 whether the translation is done by human or machine. A machine 601 would simply answer all calls to a specific address 602 (sip:translator@asp.com), and echo the media (text or speech) back 603 to the caller after conversion (conversion direction would be 604 determined by the media capabilities indicated in the INVITE). In 605 fact, there are other applications for such conversion systems. 606 Providers could not only enable services for the hearing impaired, 607 but other applications as well. Examples include voice browsing of 608 the web, e-mail to speech readout over phones, and instant message 609 to voicemail services. In fact, the opposite direction is quite 610 likely - providers that perform these services can reuse their 611 systems, without any work, to also provide services to the hearing 612 impaired, as long as a relay service is reachable across the net. 614 6.5 Hearing Impaired Calling Party through Relay 616 In this section, we consider a relay where the calling party is 617 hearing impaired. 618 This service works much like the one described above, relying on 619 third party call control mechanisms. The hearing impaired caller 620 sends an INVITE with SDP containing no codecs, targeted for the 621 called party. If the called party accepts, the caller launches an 622 INVITE to one or two Relay services (depending on whether the 623 hearing impaired caller is able to communicate verbally or not). 624 The INVITE to speech to text translation service contains SDP 625 where the caller would like to receive the text; the response 626 contains SDP that the caller places in the ACK to the called 627 party. This connects the called party with the speech to text 628 translator, with the resultant text being sent to the caller. If 629 text to speech service is also needed, the caller places the SDP 630 it received in the 200 OK from the called party into an INVITE to 631 the translator. The response contains SDP with an address where 632 the caller can send text. 634 Figure 4 shows a call flow using only speech to text translation 635 services. 637 | | | 638 | F1: INVITE no SDP | | 639 | --------------------> | | 640 | | | 641 | F2: 200 OK SDP1 | | 642 | <-------------------- | | 644 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 12 of 21] 645 | | | 646 | | | 647 | F3: INVITE | | 648 | ----------------------------------------------> | 649 | | | 650 | | F4: 200 OK SDP2 | 651 | <---------------------------------------------- | 652 | | | 653 | F5: ACK | | 654 | ----------------------------------------------> | 655 | | | 656 | | | 657 | F6: ACK SDP2 | | 658 | --------------------> | | 659 | | | 660 | | | 661 | RTP (speech) | | 662 |======================>| RTP (speech) | 663 | |========================>| 664 |<================================================| 665 | RTP (text) | | 666 | | | 667 | | | 668 | | | 669 | | | 670 | | | 671 | | | 673 Hearing Called Speech to 674 Impaired Party Text(STT)Relay 675 Caller 677 Figure 4: Hearing Impaired Caller Call Flow 679 7. General Requirements 680 It is desired among the hearing impaired user community that a 681 relay service to automatically be enabled depending on the pre-set 682 user preferences. This enables avoiding the inconvenience of 683 current relay services that requires a caller to call the relay 684 service first and then having the relay call the called party. 685 The above described call flows are only to illustrate the 686 possibilities and are not considered final call flow designs. It 687 may be a good idea work out the call flows for Relays calls, so 688 that any IP relay service such as 3G wireless can expect the pre- 689 defined scenarios. To aid the design of those scenarios, we could 690 identify a number of general requirements that relay service 691 providers and terminal manufacturers SHOULD adhere to. Keep also 692 in mind that in some countries, relay services are NOT for free as 693 in other. Another problem in some countries is that the 694 availability of relay centers is severely limited. This will be 695 reflected in some of the requirements. 697 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 13 of 21] 698 The relay enabled terminal (from now on just named terminal) MUST 699 be able to place calls transparently, the hearing impaired user 700 does not have to call the relay center first and then tell the 701 phone number to the operator who to call. It SHOULD be done 702 automatically. The terminal will connect to the relay center and 703 call the called party automatically. Very useful for ordering 704 Pizza via the telephone for example. Placing a call should be as 705 easy and comfortable for any user, including hearing and speech 706 impaired users. 708 The Relay Service providers and terminals MUST be able to 709 interwork with each other regardless the Relay Service provider or 710 the manufacturer of the (Relay enabled) terminal. This also means 711 that when a hearing impaired user from the Netherlands calls a 712 user located in the United States, that a Dutch Relay Provider or 713 an American Relay Provider CAN be used without any problem during 714 the call. This also means that modern terminals SHOULD be able to 715 support the legacy relay protocols during the first years until 716 all users have switched over to IP SIP based relay services. 718 The terminal MUST be able to receive relay calls any time and from 719 any location. This capability is already included in SIP. The 720 user preferences in the REGISTER will indicate what relay 721 requirements are desired (at minimum text support MUST be 722 supported). Upon logging in, the terminal SHOULD be able to 723 automatically download all user settings. 725 The terminal MUST be able to activate the Relay service ANYTIME 726 during a call without interrupting the call. It MUST also be 727 possible to disable the relay service during the call. 729 The terminal MUST be able to set up user preferences easily to 730 specify language, mode of relay (such as: SL/video to/from speech, 731 text to/from video or speech, also as an extended service English 732 to i.e. Spanish text, relay can cross language barriers if 733 supported). This is pre-set, but it can be overruled by one button 734 or via a short list with alternative options. The terminal and 735 relay service provider SHOULD be able to enable/deliver services 736 like a "real-time closed captioning" where the terminal receives 737 the video/audio of a caller/called, but the relay center will 738 translate the audio and display the text as subtitles. This will 739 also offer possibilities like commercial translation service (via 740 voice-over or text). 742 The terminal MUST be able to receive the correct Caller-ID of the 743 caller that calls the hearing impaired user via the relay, and not 744 the relay's Caller-ID (note: this is the caller ID and not the SIP 745 Call-ID). 747 The terminal MUST be able to specify which pre-defined numbers 748 require the use of relay service and which are direct calls (TTY 749 to TTY for text, video to video for sign language) and do not 750 require relay. 752 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 14 of 21] 753 The terminal MUST be able to download and upload the user settings 754 and the address book, which should be stored as a database file at 755 a centrally located server, which can be the relay service 756 provider or the home provider of the user. 758 It SHOULD be possible for a user to store user preferences and 759 settings on a web service, similar to the "My Yahoo" type service, 760 to allow the user access to his/her personal profile and services 761 from any web-enabled device. In case of a mobile terminal user; 762 the above described requirement SHOULD also be available via a 763 wireless web service, where available. 765 The terminal SHOULD be able to poll for the nearest Relay Proxy to 766 reduce data traffic and reduce the cost of network usage. Also the 767 terminal MUST be able to change relay service providers at any 768 time (preferably via a menu with the relay service providers 769 listed). 771 The terminal SHOULD be able to use the relay service provider list 772 to sequence the relay service providers in a preferred position: 773 which to use first for outgoing calls and automatically move on to 774 the next in the list if it is busy or does not support the 775 required services. 777 Relay service providers SHOULD be able to advertise the services 778 they have and update for new services, perhaps via a central 779 (voluntary) registration or via dialing into an info number. The 780 terminal should be able to dial automatically to such info numbers 781 or via the SIP INFO method. This would stimulate competition 782 between relay service providers, which will lead to lower service 783 prices and/or more different kind of services. 785 Relay service providers SHOULD be able to act as an answering 786 machine and provide (unified) message services. The terminal 787 SHOULD have the capability to retrieve messages (answering machine 788 mode). The terminal SHOULD have a pre-configurable setting that 789 automatically connects a calling party to the relay service 790 provider for answering machine service when the user does not want 791 to receive incoming calls. This happens transparently without 792 extra handling of the call (assumed that the called user is 793 subscribed to such a service). 795 Voice messages left for the hearing impaired user MUST be 796 interpreted at the relay service provider into the format 797 designated by the user (email, IM, video clip, etc.) and 798 transmitted to the hearing impaired user�s terminal on demand or 799 when the terminal registers. Note: This "answering service" 800 would also be a possible commercial service for hearing users. If 801 a third party unified messaging service is used by the hearing 802 impaired user; a specific relay service provider SHOULD be able to 803 be authorized to access the unified message box and translate all 805 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 15 of 21] 806 audio messages into a pre-selected form. For example voice to text 807 (for fax or e-mail or IM) or voice to video for sign language. 809 The terminal MUST be able to distinguish non-relay calls from 810 relay calls and direct TTY calls, if this number is NOT listed in 811 the address book (which stores the number and options for direct 812 terminal to terminal calls such as TTY to TTY, which do not 813 require a relay). A message will be sent to the originator of the 814 call (183 Hearing impaired user RELAY call only). Depending on 815 technological advances and the user's preferences, instead of the 816 terminal returning a 183 message, the terminal MAY connect to the 817 relay service provider and accept the call as if there is NO relay 818 service provider in between. 820 The Relay Service provider SHOULD be able to offer the STEALTH 821 mode; invisible relay services for the hearing impaired caller, in 822 this way the user can conceal his/her hearing impairment). 823 This may be seen as a separate service and possibly charged extra 824 due the speed of accepting the call (minimal delay on picking-up) 825 and extra effort to be invisible to the caller. Special hardware 826 and software may be required for computer run speech to text 827 conversion (this may require specialized relay service providers). 828 Note: this service can be used selectively for certain callers, 829 and other callers are just notified by the 183 message. 831 The terminal and the Relay Service Provider MUST be able to allow 832 calls to be placed anonymously, so that the called user cannot see 833 who is calling. The Relay Service provider MUST in this case keep 834 the identity of the calling user confidential as described in 7. 836 When starting a relay session; the Relay Service Provider MUST 837 show the charges of the relaying service per minute. The user MUST 838 be able to refuse the service if the charges are too high or opt 839 for a cheaper service (use text instead of the video). 841 An independent payment service provider SHOULD be used, so that 842 any relay service provider is assured of payment of the charges. 843 This requires that the hearing impaired users use some form of 844 subscription for relay services that require extra features. This 845 leaves room for subsidizing the hearing impaired users, for 846 example a monthly pre-paid amount will be deposited to the relay 847 account. If a user uses more then the pre-paid amount, the user 848 will be billed for the additional charges. 850 Wireless telephony systems (GSM, UMTS, CDMA2000, cellular systems 851 etc) MUST enable the wireless phones to connect to relay service 852 providers. This can be done directly via wireless IP connections 853 OR via a special relay node. This node MUST be able to translate 854 circuit switched (GSM, etc) relay into IP relay. 856 The Relay Service provider MAY also offer remote interpreting, the 857 terminal MAY be modified to enabled this service. This is 859 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 16 of 21] 860 desirable in countries experiencing a shortage of Interpreters. 861 And there are many situations where it just requires a short time 862 of interpreting. 864 7. Security Considerations 866 Because an interpreter is generally required when a hearing 867 impaired individual has a conversation with a hearing individual, 868 whether in person or using a medium such as the telephone, 869 interpreters are privy to a great deal of private information. 870 For this reason, both the Deaf and interpreter communities are 871 vitally interested in the ethics and professionalism of the 872 interpreter. For example, interpreters in the United States that 873 are certified by the organizations recognized by the American Deaf 874 Community, such as the National Association of the Deaf (NAD[16]) 875 and the Registry of Interpreters for the Deaf (RID[17]), are 876 required as part of their certification to support the Codes of 877 Ethics of their organizations. Other countries may also have 878 pertinent legislation. 880 Hence these requirements: 882 All relay operators and other interpreters or organizations 883 involved in relaying calls SHALL be required to subscribe to a 884 generally accepted Code of Ethics for interpreters. As an example, 885 the Code of Ethics required for membership in RID is as follows: 887 The Registry of Interpreters for the Deaf, Inc. has set forth the 888 following principles of ethical behavior to protect and guide 889 interpreters and transliterators and hearing and deaf consumers. 890 Underlying these principles is the desire to insure for all the 891 right to communicate. 893 This Code of Ethics applies to all members of the Registry of 894 Interpreters for the Deaf, Inc. and to all certified non-members. 896 Interpreters/transliterators shall keep all assignment-related 897 information strictly confidential. 899 Interpreters/transliterators shall render the message faithfully, 900 always conveying the content and spirit of the speaker using 901 language most readily understood by the person(s) whom they serve. 903 Interpreters/transliterators shall not counsel, advise or 904 interject personal opinions. 906 Interpreters/transliterators shall accept assignments using 907 discretion with regard to skill, setting, and the consumers 908 involved. 910 Interpreters/transliterators shall request compensation for 911 services in a professional and judicious manner. 913 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 17 of 21] 914 Interpreters/transliterators shall function in a manner 915 appropriate to the situation. 917 Interpreters/transliterators shall strive to further knowledge and 918 skills through participation in workshops, professional meetings, 919 interaction with professional colleagues, and reading of current 920 literature in the field. 922 Interpreters/transliterators, by virtue of membership or 923 certification by the RID, Inc., shall strive to maintain high 924 professional standards in compliance with the Code of Ethics. 926 This Code of Ethics is widely accepted and supported as a standard 927 within the American Deaf community and the American community of 928 interpreters. More information on RID can be found from the 929 organization�s web site, see reference [16]. 931 In addition to the standards required for individual relay 932 operators, interpreters, and companies that provide relay 933 services, the actual transmissions MUST be secured. 935 An extension to the requirement for "STEALTH": relay 936 operators/interpreters MAY act on behalf of the user, at the 937 request of the user. For example, the user can ask the operator 938 to call a company to file a complaint. This requires 939 confidentially and an extension to the usual role of an 940 interpreter. 942 9. References and Footnotes 944 [1] M. Handley, H. Schulzrinne, E. Schooler, and J. Rosenberg, 945 "SIP: session initiation protocol," Request for Comments 2543, 946 Internet Engineering Task Force, Mar. 1999. 948 [2] M. Handley and V. Jacobson. "SDP: Session Description 949 Protocol." Request for Comments 2327, Internet Engineering Task 950 Force,April 1998. 952 [3] H. Schulzrinne and J. Rosenberg, "SIP caller preferences and 953 callee capabilities," Internet Draft, Internet Engineering Task 954 Force, June 2001. Work in progress. 956 [4] J. Rosenberg, H. Schulzrinne, J. Peterson and G. Camarillo, 957 "Third party call control in SIP," Internet Draft, Internet 958 Engineering Task Force, Mar. 2001. Work in progress. 960 [5] G. Hellstrom, "RTP payload for text conversation," Request for 961 Comments 2793, Internet Engineering Task Force, May 2000. 963 [6] J. Rosenberg, H. Schulzrinne and H. Sinnreich, "SIP Enabled 964 Services to Support the Hearing Impaired" Internet Draft, Internet 965 Engineering Task Force, July 2000. Work in progress. 967 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 18 of 21] 969 [7] C. Gearhart, A. Van Wijk and H. Sinnreich, "A Proposed Set of 970 requirements for SIP Support for Deaf or Speech Impaired 971 Customers" Internet Draft, Internet Engineering Task Force, 972 November 2000. Work in progress. 974 [8] S. Bradner, "Key words for use in RFCs to indicate requirement 975 levels". Request for Comments 2119, Internet Engineering Task 976 Force. March 1997. 978 [9] TTY: an acronym for a text telephone device used by Deaf 979 individuals to communicate via telephone systems; commonly 980 referred to as a TDD by the hearing community. 982 [10] Bradner, S., "The Internet Standards Process -- Revision 3", 983 BCP9, RFC 2026, October 1996. 985 [11] Baker, Charlotte, and Robbin Battison. "Sign Language and 986 the Deaf Community: Essays in Honor of William Stokoe". National 987 Association of the Deaf, June 1980. 989 [12] Moore, Matthew, et al. "For Hearing People Only: Answers to 990 Some of the Most Commonly Asked Questions About the Deaf 991 Community, Its Culture, and the Deaf Reality". MSM Productions 992 Ltd., 2nd Edition, September 1993. 994 [13] Padden, Carol, and Tom Humphries. "Deaf in America: Voices 995 from a Culture". Harvard University Press, Reprint September 1990. 997 [14] Stokoe, William. "Sign and Culture: A Reader for Students 998 of American Sign Language". Linstok Press, June 1980. 1000 [15] J. Rosenberg, R. Sparks, D. Willis, B. Campbell, H. 1001 Schulzrinne, J. Lennox, C. Huitema, B. Aboba, D. Gurle and D. 1002 Oran, "SIP extensions for instant messaging," Internet Draft, 1003 Internet Engineering Task Force, April 2001. Work in progress. 1005 [16] National Association of the Deaf. A national organization 1006 of, for, and operated by Americans who are Deaf or deaf. 1007 Organized in 1880, it is "the oldest and largest organization 1008 representing people with disabilities in the United States. The 1009 NAD safeguards the accessibility and civil rights of 28 million 1010 deaf and hard of hearing Americans in a variety of areas including 1011 education, employment, health care and social services, and 1012 telecommunications. A private, non-profit 501(c)(3) organization, 1013 the NAD is a dynamic federation of 51 state association 1014 affiliates, sponsoring and organizational affiliates, and direct 1015 members." See "www.nad.org" for more information on this 1016 organization. 1018 [17] Registry of Interpreters for the Deaf (RID). A national, 1019 professional organization of interpreters and transliterators for 1020 the Deaf in America. "The philosophy of RID is that excellence in 1021 the delivery of interpretation and transliteration services among 1023 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 19 of 21] 1024 people who are Deaf, or Hard of Hearing, and people who are 1025 hearing, will ensure effective communication. As the professional 1026 association for interpreters and transliterators, the RID serves 1027 as an essential arena for its members in their pursuit of 1028 excellence. It is the mission of the Registry of Interpreters for 1029 the Deaf, Inc., to provide international, national, regional, 1030 state, and local forums and an organizational structure for the 1031 continued growth and development of the professions of 1032 interpretation and transliteration of American Sign Language and 1033 English." See "www.rid.org" for detailed information on this 1034 organization. 1036 10 Acknowledgments 1038 The authors would like to thank the following deaf individuals, 1039 professional interpreters, and others who have contributed to the 1040 development of this document: 1041 Vint Cerf/WCOM. 1042 Teresa Hastings/WCOM. 1043 Charles Estes/WCOM. 1044 Helene Cohen-Gilbert, Coordinator, Collin County Community 1045 College, Texas, USA, Interpreter Preparation Program � Deaf. 1046 Nathan Charlton, Royal National Institute for Deaf People (RNID), 1047 London, United Kindom. 1048 Grant Laird. 1049 Brenden Gilbert. 1051 11. Author's Addresses 1053 Arnoud van Wijk 1054 Ericsson EuroLab Netherlands BV 1055 P.O. Box 8 1056 5120 AA Rijen 1057 The Netherlands 1058 Fax: +31-161-247569 1059 email: Arnoud.van.Wijk@eln.ericsson.se 1061 Jonathan Rosenberg 1062 Dynamicsoft 1063 72 Eagle Rock Avenue 1064 First Floor 1065 East Hanover, NJ 07936 1066 email: jdrosen@dynamicsoft.com 1068 Cathy Gearhart 1069 Ericsson, Inc. 1070 P.O. Box 833675, M/S L-04 1071 Richardson, TX 75083-3875 1072 email: cathy.gearhart@ericsson.com 1074 Henry Sinnreich 1075 Worldcom 1076 400 International Parkway 1078 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 20 of 21] 1079 Richardson, Texas 75081 1080 email: henry.sinnreich@wcom.com 1082 Henning Schulzrinne 1083 Columbia University 1084 M/S 0401 1085 1214 Amsterdam Ave. 1086 New York, NY 10027-7003 1087 email: schulzrinne@cs.columbia.edu 1089 Full Copyright Statement 1091 "Copyright (C) The Internet Society 2001. All Rights Reserved. 1092 This document and translations of it may be copied and furnished 1093 to others, and derivative works that comment on or otherwise 1094 explain it or assist in its implmentation may be prepared, copied, 1095 published and distributed, in whole or in part, without 1096 restriction of any kind, provided that the above copyright notice 1097 and this paragraph are included on all such copies and derivative 1098 works. However, this document itself may not be modified in any 1099 way, such as by removing the copyright notice or references to the 1100 Internet Society or other Internet organizations, except as needed 1101 for the purpose of developing Internet standards in which case the 1102 procedures for copyrights defined in the Internet Standards 1103 process must be followed, or as required to translate it into 1104 languages other than English. The limited permissions granted 1105 above are perpetual and will not be revoked by the Internet 1106 Society or its successors or assigns. 1107 This document and the information contained herein is provided on 1108 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 1109 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 1110 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1111 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1112 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1114 van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne [Page 21 of 21]