idnits 2.17.1 draft-ietf-codec-guidelines-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 10, 2012) is 4483 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 3979 (ref. 'IPR') (Obsoleted by RFC 8179) -- Obsolete informational reference (is this intentional?): RFC 3356 (ref. 'COLLAB') (Obsoleted by RFC 6756) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group JM. Valin 3 Internet-Draft Mozilla 4 Intended status: Informational S. Borilin 5 Expires: July 13, 2012 SPIRIT DSP 6 K. Vos 8 C. Montgomery 9 Xiph.Org Foundation 10 R. Chen 11 Broadcom Corporation 12 January 10, 2012 14 Guidelines for Development of an Audio Codec Within the IETF 15 draft-ietf-codec-guidelines-08 17 Abstract 19 This document provides general guidelines for work on developing and 20 specifying an interactive audio codec within the IETF. These 21 guidelines cover the development process, evaluation, requirements 22 conformance, and intellectual property issues. 24 Status of this Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on July 13, 2012. 41 Copyright Notice 43 Copyright (c) 2012 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Development Process . . . . . . . . . . . . . . . . . . . . . 4 60 3. Evaluation, Testing, and Characterization . . . . . . . . . . 7 61 4. Specifying the Codec . . . . . . . . . . . . . . . . . . . . . 9 62 5. Intellectual Property . . . . . . . . . . . . . . . . . . . . 11 63 6. Relationship with Other SDOs . . . . . . . . . . . . . . . . . 14 64 7. Security Considerations . . . . . . . . . . . . . . . . . . . 16 65 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 66 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 18 67 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19 68 10.1. Normative References . . . . . . . . . . . . . . . . . . 19 69 10.2. Informative References . . . . . . . . . . . . . . . . . 19 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 72 1. Introduction 74 This document describes a process for work in the IETF codec WG on 75 standardization of an audio codec that is optimized for use in 76 interactive Internet applications and that can be widely implemented 77 and easily distributed among application developers, service 78 operators, and end users. 80 2. Development Process 82 The process outlined here is intended to make the work on a codec 83 within the IETF transparent, predictable, and well organized, in a 84 way that is consistent with [PROCESS]. Such work might involve 85 development of a completely new codec, adaptation of an existing 86 codec to meet the requirements of the working group, or integration 87 between two or more existing codecs that results in an improved codec 88 combining the best aspects of each. To enable such procedural 89 transparency, the contributor of an existing codec must be willing to 90 cede change control to the IETF and should have sufficient knowledge 91 of the codec to assist in the work of adapting it or applying some of 92 its technology to the development or improvemnet of other codecs. 93 Furthermore, contributors need to be aware that any codec that 94 results from work within the IETF is likely to be different from any 95 existing codec that was contributed to the Internet Standards 96 Process. 98 Work on an interactive audio codec development is expected to proceed 99 as follows: 101 1. IETF participants will identify the requirements to be met by an 102 Internet codec, in the form of an Internet-Draft. 104 2. Interested parties are encouraged to make contributions proposing 105 existing or new codecs, or elements thereof, to the codec WG as 106 long as these contributions are within the scope of the WG. 107 Ideally, these contributions should be in the form of Internet 108 Drafts, although other forms of contributions are also possible, 109 as discussed in [PROCESS]. 111 3. Given the importance of IPR to the activities of the working 112 group, any IPR disclosures must be made in a timely way. 113 Contributors are required, as described in [IPR], to disclose any 114 known IPR, both first and third party. Timely disclosures are 115 particularly important, since those disclosures may be material 116 to the decision process of the working group. 118 4. As contributions are received and discussed within the working 119 group, the group should gain a clearer understanding of what is 120 achievable within the design space. As a result, the authors of 121 the requirements document should iteratively clarify and improve 122 their document to reflect the emerging working group consensus. 123 This is likely to involve collaboration with IETF working groups 124 in other areas, such as collaboration with working groups in the 125 Transport area to identify important aspects of packet 126 transmission over the Internet and to understand the degree of 127 rate adaptation desirable, and with working groups in the RAI 128 area to ensure that information about and negotiation of the 129 codec can be easily represented at the signaling layer. In 130 parallel with this work, interested parties should evaluate the 131 contributions at a higher level to see which requirements might 132 be met by each codec. 134 5. Once a sufficient number of proposals has been received, the 135 interested parties will identify the strengths, weaknesses, and 136 innovative aspects of the contributed codecs. This step will 137 consider not only the codecs as a whole, but also key features of 138 the individual algorithms (predictors, quantizers, transforms, 139 etc.). 141 6. Interested parties are encouraged to collaborate together and 142 combine the best ideas from the various codec contributions into 143 a consolidated codec definition, representing the merging of some 144 of the contributions. Through this iterative process, the number 145 of proposals will reduce and consensus will generally form around 146 one of them. At that point, the working group should adopt that 147 document as a working group item, forming a baseline codec. 149 7. IETF participants should then attempt to iteratively add to or 150 improve each component of the baseline codec reference 151 implementation, where by "component" we mean individual 152 algorithms such as predictors, transforms, quantizers, and 153 entropy coders. The participants should proceed by trying new 154 designs, applying ideas from the contributed codecs, evaluating 155 "proof of concept" ideas, and using their expertise in codec 156 development to improve the baseline codec. Any aspect of the 157 baseline codec might be changed (even the fundamental principles 158 of the codec) or the participants might start over entirely by 159 scrapping the baseline codec and designing a completely new one. 160 The overriding goal shall be to design a codec that will meet the 161 requirements defined in the requirements document 162 [codec-requirements]. Given the IETF's open standards process, 163 any interested party will be able to contribute to this work, 164 whether or not they submitted an Internet-Draft for one of the 165 contributed codecs. The codec itself should be normatively 166 specified with code in an Internet-Draft. 168 8. In parallel with work on the codec reference implementation, 169 developers and other interested parties should perform evaluation 170 of the codec as described under Section 3, IETF participants 171 should define (within the PAYLOAD Working Group) the codec's 172 payload format for use with the Real-time Transport Protocol 173 [RTP]. Ideally, application developers should test the codec by 174 implementing it in code and deploying it in actual Internet 175 applications. Unfortunately, developers will frequently wait 176 until RFC or until a stable bitstream is guaranteed before 177 deployment. As such, this is a nice-to-have and not a 178 requirement for this process. Lab implementations are certainly 179 encouraged. 181 9. The group will produce a testing results document. The document 182 will be a living document that captures testing done before the 183 codec stabilized, after it has stabilized, and after the codec 184 specification is issued as an RFC. The document serves the 185 purpose of helping the group determine whether the codec meets 186 the requirements. Any testing done after the codec RFC is issued 187 helps implementors understand the final performance of the codec. 188 The process of testing is described in Section 3. 190 3. Evaluation, Testing, and Characterization 192 Lab evaluation of the codec being developed should happen throughout 193 the development process because it will help ensure that progress is 194 being made toward fulfillment of the requirements. There are many 195 ways in which continuous evaluation can be performed. For minor, 196 uncontroversial changes to the codec it should usually be sufficient 197 to use objective measurements (e.g., PESQ [ITU-T P.862], PEAQ [ITU-R 198 BS.1387-1], and segmental signal-to-noise ratio) validated by 199 informal subjective evaluation. For more complex changes (e.g., when 200 psychoacoustic aspects are involved) or for controversial issues, 201 internal testing should be performed. An example of internal testing 202 would be to have individual participants rate the decoded samples 203 using one of the established testing methodologies, such as [ITU-R 204 BS.1534] (MUSHRA). 206 Throughout the process, it will be important to make use of the 207 Internet community at large for real-world distributed testing. This 208 will enable many different people with different equipment and use 209 cases to test the codec and report any problems they experience. In 210 the same way, third-party developers will be encouraged to integrate 211 the codec into their software (with a warning about the bit-stream 212 not being final) and provide feedback on its performance in real- 213 world use cases. 215 Characterization of the final codec must be based on the reference 216 implementation only (and not on any "private implementation"). This 217 can be performed by independent testing labs or, if this is not 218 possible, using the testing labs of the organizations that contribute 219 to the Internet Standards Process. Packet loss robustness should be 220 evaluated using actual loss patterns collected from use over the 221 Internet, rather than theoretical models. The goals of the 222 characterization phase are to: 224 o ensure that the requirements have been fulfilled 226 o guide the IESG in its evaluation of the resulting work 228 o assist application developers in understanding whether the codec 229 is suitable for a particular application 231 The exact methodology for the characterization phase can be 232 determined the working group. Because the IETF does not have testing 233 resources of its own, it has to rely on the resources of its 234 participants. For this reason, even if the group agrees that a 235 particular test is important, if no one volunteers to do it, or if 236 volunteers do not complete it in a timely fashion, then that test 237 should be discarded. This ensures that only important tests be done, 238 and in particular those tests which are important to participants. 240 4. Specifying the Codec 242 Specifying a codec requires careful consideration around what is 243 required vs. what is left to the implementation. The following text 244 provides guidelines for consideration by the working group: 246 1. Any audio codec specified by the codec Working Group must include 247 source code for a normative software implementation, documented 248 in an Internet Draft destined for standards track RFC. This 249 implementation will be used to verify conformance of an 250 implementation. Although a text description of the algorithm 251 should be provided, its use should be limited to helping the 252 reader in understanding the source code. Should the description 253 contradict the source code, the latter shall take precedence. 254 For convenience, the source code may be provided in compressed 255 form, with base64 [BASE64] encoding. 257 2. Because of the size and complexity of most codecs, it is possible 258 that even after publishing the RFC, bugs will be found in the 259 reference implementation, or differences between the 260 implementation and the text description. As usual, an errata of 261 the RFC should be maintained. Although a public software 262 repository containing the current reference implementation is 263 desirable, the normative implementation would still be the RFC. 265 3. It is the intention of the group to allow the greatest possible 266 choice of freedom in implementing the specification. 267 Accordingly, the number of binding RFC2119 keywords will be the 268 minimum that still allows interoperable implementations. In 269 practice this generally means that only the decoder needs to be 270 normative, so that the encoder can improve over time. This also 271 enables different trade-offs between quality and complexity. 273 4. To reduce the risk of bias towards certain CPU/DSP architectures, 274 ideally the decoder specification should not require "bit-exact" 275 conformance with the reference implementation. In that case, the 276 output of a decoder implementation should only be "close enough" 277 to the output of the reference decoder and a comparison tool 278 should be provided along with the codec to verify objectively 279 that the output of a decoder is likely to be perceptually 280 indistinguishable from that of the reference decoder. An 281 implementation may still wish to produce an output that is bit- 282 exact with the reference implementation to simplify the testing 283 procedure. 285 5. To ensure freedom of implementation, decoder-side only error 286 concealment does not need to be specified, although the reference 287 implementation should include the same packet loss concealment 288 (PLC) algorithm as used in the testing phase. Is it up to the 289 working group to decide whether minimum requirements on PLC 290 quality will be required for compliance with the specification. 291 Obviously, any information signaled in the bitstream intended to 292 aid PLC needs to be specified. 294 6. An encoder implementation should not be required to make use of 295 all the "features" (tools) in the bit-stream definition. 296 However, the codec specification may require that an encoder 297 implementation be able to generate any possible bit-rate. Unless 298 a particular "profile" is defined in the specification, the 299 decoder must be able to decode all features of the bit-stream. 300 The decoder must also be able to handle any combination of bits, 301 even combinations that cannot be generated by the reference 302 encoder. It is recommended that the decoder specification shall 303 define how the decoder should react to "impossible" packets (e.g. 304 reject, consider as valid). However, an encoder must never 305 generate such packets that do not conform to the bit-stream 306 definition. 308 7. Compressed test vectors should be provided as a means to verify 309 conformance with the decoder specification. These test vectors 310 should be designed to exercise as much of the decoder code as 311 possible. 313 8. While the exact encoder will not be specified, it is recommended 314 to specify objective measurement targets for an encoder, below 315 which use of a particular encoder implementation is not 316 recommended. For example, one such specification could be: "the 317 use of an encoder whose PESQ MOS is better than 0.1 below the 318 reference encoder in the following conditions is not 319 recommended". 321 5. Intellectual Property 323 Producing an unencumbered codec is desirable for the following 324 reasons: 326 o It is the experience of a wide variety of application developers 327 and service providers that encumbrances such as licensing and 328 royalties make it difficult to implement, deploy, and distribute 329 multimedia applications for use by the Internet community. 331 o It is beneficial to have low-cost options whenever possible, 332 because innovation - the hallmark of the Internet - is hampered 333 when small development teams cannot deploy an application because 334 of usage-based licensing fees and royalties. 336 o Many market segments are moving away from selling hard-coded 337 hardware devices and toward freely distributing end-user software; 338 this is true of numerous large application providers and even 339 telcos themselves. 341 o Compatibility with the licensing of typical open source 342 applications implies the need to avoid encumbrances, including 343 even the requirement to obtain a license for implementation, 344 deployment, or use (even if the license does not require the 345 payment of a fee). 347 Therefore, a codec that can be widely implemented and easily 348 distributed among application developers, service operators, and end 349 users is preferred. Many existing codecs that might fulfill some or 350 most of the technical attributes listed above are encumbered in 351 various ways. For example, patent holders might require that those 352 wishing to implement the codec in software, deploy the codec in a 353 service, or distribute the codec in software or hardware need to 354 request a license, enter into a business agreement, pay licensing 355 fees or royalties, or adhere to other special conditions or 356 restrictions. Because such encumbrances have made it difficult to 357 widely implement and easily distribute high-quality codecs across the 358 entire Internet community, the working group prefers unencumbered 359 technologies in a way that is consistent with BCP 78 and BCP 79. In 360 particular, the working group shall heed the preference stated in BCP 361 79: "In general, IETF working groups prefer technologies with no 362 known IPR claims or, for technologies with claims against them, an 363 offer of royalty-free licensing." Although this preference cannot 364 guarantee that the working group will produce an unencumbered codec, 365 the working group shall follow BCP 79, and adhere to the spirit of 366 BCP 79. The working group cannot explicitly rule out the possibility 367 of adopting encumbered technologies; however, the working group will 368 try to avoid encumbered technologies that require royalties or other 369 encumbrances that would prevent such technologies from being easy to 370 redistribute and use. 372 When considering license terms for technologies with IPR claims 373 against them, some members of the working group have expressed their 374 preference for license terms which: 376 o are available to all, worldwide, whether or not they are working 377 group participants 379 o extend to all essential claims owned or controlled by the licensor 381 o do not require payment of royalties, fees or other consideration 383 o do not require licensees to adhere to restrictions on usage 384 (though, licenses which apply only to implementation of the 385 standard are acceptable) 387 o do not otherwise impede the ability of the codec to be implemented 388 in open-source software projects 390 The following guidelines will help to maximize the odds that the 391 codec will be unencumbered: 393 1. In accordance with BCP 79 [IPR], contributed codecs should 394 preferably use technologies with no known IPR claims or 395 technologies with an offer of royalty-free licensing. 397 2. As described in BCP 79, the working group should use technologies 398 that are perceived by the participants to be safer with regard to 399 IPR issues. 401 3. Contributors must disclose IPR as specified in BCP 79. 403 4. In cases where no royalty-free license can be obtained regarding 404 a patent, BCP 79 suggests that the working group consider 405 alternative algorithms or methods, even if they result in lower 406 quality, higher complexity, or otherwise less desirable 407 characteristics. 409 5. In accordance with BCP 78 [TRUST], the source code for the 410 reference implementation must be made available under a BSD-style 411 license (or whatever license is defined as acceptable by the IETF 412 Trust when the Internet-Draft defining the reference 413 implementation is published). 415 Many IPR licenses specify that a license is granted only for 416 technologies which are adopted by the IETF as a standard. While 417 reasonable, this has the unintended side-effect of discouraging 418 implementation prior to RFC status. Real-world implementation is 419 beneficial for evaluation of the codec. As such, entities making IPR 420 license statements are encouraged to use wording which permits early 421 implementation and deployment. 423 IETF participants should be aware that, given the way patents work in 424 most countries, the resulting codec can never be guaranteed to be 425 free of patent claims because some patents may not be known to the 426 contributors, some patent applications may not be disclosed at the 427 time the codec is developed, and only courts of law can determine the 428 validity and breadth of patent claims. However, these observations 429 are no different within the Internet Standards Process than they are 430 for standardization of codecs within other SDOs (or development of 431 codecs outside the context of any SDO), and furthermore are no 432 different for codecs than for other technologies worked on within the 433 IETF. In all these cases, the best approach is to minimize the risk 434 of unknowingly incurring encumbrance on existing patents. Despite 435 these precautions, participants need to understand that, practically 436 speaking, it is nearly impossible to _guarantee_ that implementers 437 will not incur encumbrance on existing patents. 439 6. Relationship with Other SDOs 441 It is understood that other SDOs are also involved in the codec 442 development and standardization, including but not necessarily 443 limited to: 445 o The Telecommunication Standardization Sector (ITU-T) of the 446 International Telecommunication Union (ITU), in particular Study 447 Group 16 449 o The Moving Picture Experts Group (MPEG) of the International 450 Organization for Standardization and International 451 Electrotechnical Commission (ISO/IEC) 453 o The European Telecommunications Standards Institute (ETSI) 455 o The 3rd Generation Partnership Project (3GPP) 457 o The 3rd Generation Partnership Project 2 (3GPP2) 459 It is important to ensure that such work does not constitute 460 uncoordinated protocol development, of the kind described in 461 [UNCOORD] in the following principle: 463 [T]he IAB considers an essential principle of the protocol 464 development process that only one SDO maintains design authority 465 for a given protocol, with that SDO having ultimate authority over 466 the allocation of protocol parameter code-points; defining the 467 intended semantics, interpretation, and actions associated with 468 those code-points. 470 The work envisioned by this guidelines document is not uncoordinated 471 in the sense described in the foregoing quote, since the intention of 472 this process is that two possible outcomes might occur: 474 1. The IETF adopts an existing audio codec, and specifies that it is 475 the "anointed" IETF Internet codec. In such a case, codec 476 ownership lies entirely with the SDO which produced the codec, 477 and not with the IETF, OR 479 2. The IETF produces a new codec. Even if this codec uses concepts, 480 algorithms, or even source code from a codec produced by another 481 SDO, the IETF codec is a specification unto itself and under 482 complete control of the IETF. Any changes or enhancements made 483 by the original SDO to the codecs whose components the IETF used 484 are not applicable to the IETF codec. Such changes would be 485 incorporated as a consequence of a revision or extension of the 486 IETF RFC. In no case should the new codec reuse a name or code 487 point from another SDO. 489 Although there is already sufficient codec expertise available among 490 IETF participants to complete the envisioned work, additional 491 contributions are welcome within the framework of the Internet 492 Standards Process, in the following ways: 494 o Individuals who are technical contributors to codec work within 495 other SDOs can participate directly in codec work within the IETF. 497 o Other SDOs can contribute their expertise (e.g., codec 498 characterization and evaluation techniques) and thus facilitate 499 the testing of a codec produced by the IETF. 501 o Any SDO can provide input to IETF work through liaison statements. 503 However, it is important to note that final responsibility for the 504 development process and the resulting codec will remain with the IETF 505 as governed by BCP 9 [PROCESS]. 507 Finally, there is precedent for the contribution of codecs developed 508 elsewhere to the ITU-T (e.g., AMR Wideband was standardized 509 originally within 3GPP). This is a model to explore as the IETF 510 coordinates further with the ITU-T in accordance with the 511 collaboration guidelines defined in [COLLAB]. 513 7. Security Considerations 515 The procedural guidelines for codec development do not have security 516 considerations. However, the resulting codec needs to take 517 appropriate security considerations into account, as outlined in 518 [SECGUIDE] and in the security considerations of 519 [codec-requirements]. More specifically, the resulting codec must 520 avoid being subject to denial of service [DOS] and buffer overflows, 521 and should take into consideration the impact of VBR [srtp-vbr]. 523 8. IANA Considerations 525 This document has no actions for IANA. 527 9. Acknowledgments 529 We would like to thank all the other people who contributed directly 530 or indirectly to this document, including Jason Fischl, Gregory 531 Maxwell, Alan Duric, Jonathan Christensen, Julian Spittka, Michael 532 Knappe, Timothy B. Terriberry, Christian Hoene, Stephan Wenger and 533 Henry Sinnreich. We also like to thank Cullen Jennings and Gregory 534 Lebovitz for their advice. Special thanks to Peter Saint-Andre, who 535 originally co-authored this document. 537 10. References 539 10.1. Normative References 541 [IPR] Bradner, S., "Intellectual Property Rights in IETF 542 Technology", BCP 79, RFC 3979, March 2005. 544 [PROCESS] Bradner, S., "The Internet Standards Process -- Revision 545 3", BCP 9, RFC 2026, October 1996. 547 [TRUST] Bradner, S. and J. Contreras, "Rights Contributors Provide 548 to the IETF Trust", BCP 78, RFC 5378, November 2008. 550 10.2. Informative References 552 [COLLAB] Fishman, G. and S. Bradner, "Internet Engineering Task 553 Force and International Telecommunication Union - 554 Telecommunications Standardization Sector Collaboration 555 Guidelines", RFC 3356, August 2002. 557 [DOS] Handley, M., Rescorla, E., and IAB, "Internet Denial-of- 558 Service Considerations", RFC 4732, December 2006. 560 [RTP] Schulzrinne, H., Casner, S., Frederick, R., and V. 561 Jacobson, "RTP: A Transport Protocol for Real-Time 562 Applications", STD 64, RFC 3550, July 2003. 564 [BASE64] Josefsson, S., "The Base16, Base32, and Base64 Data 565 Encodings", RFC 4648, October 2006. 567 [SECGUIDE] 568 Rescorla, E. and B. Korver, "Guidelines for Writing RFC 569 Text on Security Considerations", BCP 72, RFC 3552, 570 July 2003. 572 [codec-requirements] 573 Valin, JM. and K. Vos, "Requirements for an Internet Audio 574 Codec", RFC 6366, August 2011. 576 [srtp-vbr] 577 Perkins, C. and JM. Valin, "Guidelines for the use of 578 Variable Bit Rate Audio with Secure RTP", 579 draft-ietf-avtcore-srtp-vbr (work in progress), July 2011. 581 [UNCOORD] Bryant, S. and M. Morrow, "Uncoordinated Protocol 582 Development Considered Harmful", RFC 5704, November 2009. 584 [ITU-R BS.1534] 585 "Method for the subjective assessment of intermediate 586 quality levels of coding systems", ITU-R 587 Recommendation BS.1534, January 2003. 589 [ITU-T P.862] 590 "Perceptual evaluation of speech quality (PESQ): An 591 objective method for end-to-end speech quality assessment 592 of narrow-band telephone networks and speech codecs", 593 ITU-T Recommendation P.862, October 2007. 595 [ITU-R BS.1387-1] 596 "Method for objective measurements of perceived audio 597 quality", ITU-R Recommendation BS.1387-1, November 2001. 599 Authors' Addresses 601 Jean-Marc Valin 602 Mozilla 603 650 Castro Street 604 Mountain View, CA 94041 605 USA 607 Email: jmvalin@jmvalin.ca 609 Slava Borilin 610 SPIRIT DSP 612 Email: borilin@spiritdsp.net 614 Koen Vos 616 Email: koen.vos@skype.net 618 Christopher Montgomery 619 Xiph.Org Foundation 621 Email: xiphmont@xiph.org 623 Raymond (Juin-Hwey) Chen 624 Broadcom Corporation 626 Email: rchen@broadcom.com