idnits 2.17.1 draft-ietf-issll-isslow-rtf-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 4 instances of too long lines in the document, the longest one being 7 characters in excess of 72. ** The abstract seems to contain references ([2], [5], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 303 has weird spacing: '... * As the F...' == Line 304 has weird spacing: '...require any ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 1999) is 9133 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '3' is defined on line 498, but no explicit reference was found in the text == Outdated reference: A later version (-06) exists of draft-ietf-issll-isslow-05 ** Downref: Normative reference to an Informational draft: draft-ietf-issll-isslow (ref. '1') ** Obsolete normative reference: RFC 1717 (ref. '2') (Obsoleted by RFC 1990) -- No information found for draft-andrades-framing-ext - is the name correct? -- Possible downref: Normative reference to a draft: ref. '4' == Outdated reference: A later version (-06) exists of draft-ietf-issll-isslow-mcml-05 ** Obsolete normative reference: RFC 1548 (ref. '6') (Obsoleted by RFC 1661) ** Obsolete normative reference: RFC 1549 (ref. '7') (Obsoleted by RFC 1662) Summary: 12 errors (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Carsten Bormann 3 Expires: October 1999 Universitaet Bremen TZI 4 April 1999 6 PPP in a real-time oriented HDLC-like framing 7 draft-ietf-issll-isslow-rtf-04.txt 9 Status of this memo 11 This document is an Internet-Draft and is in full conformance with 12 all provisions of Section 10 of RFC2026. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six months 20 and may be updated, replaced, or obsoleted by other documents at any 21 time. It is inappropriate to use Internet- Drafts as reference 22 material or to cite them other than as "work in progress." 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html 30 Abstract 32 A companion document describes an architecture for providing 33 integrated services over low-bitrate links, such as modem lines, ISDN 34 B-channels, and sub-T1 links [1]. The main components of the 35 architecture are: a real-time encapsulation format for asynchronous 36 and synchronous low-bitrate links, a header compression architecture 37 optimized for real-time flows, elements of negotiation protocols used 38 between routers (or between hosts and routers), and announcement 39 protocols used by applications to allow this negotiation to take 40 place. 42 This document proposes the suspend/resume-oriented solution for the 43 real-time encapsulation format part of the architecture. The general 44 approach is to start from the PPP Multilink fragmentation protocol 45 [2] and its multi-class extension [5] and add suspend/resume in a way 46 that is as compatible to existing hard- and firmware as possible. 48 1. Introduction 50 As an extension to the ``best-effort'' services the Internet is well- 51 known for, additional types of services (``integrated services'') 52 that support the transport of real-time multimedia information are 53 being developed for, and deployed in the Internet. 55 The present document defines the suspend/resume-oriented solution for 56 the real-time encapsulation format part of the architecture. As 57 described in more detail in the architecture document, a real-time 58 encapsulation format is required as, e.g., a 1500 byte packet on a 59 28.8 kbit/s modem link makes this link unavailable for the 60 transmission of real-time information for about 400 ms. This adds a 61 worst-case delay that causes real-time applications to operate with 62 round-trip delays on the order of at least a second -- unacceptable 63 for real-time conversation. 65 A true suspend/resume-oriented approach can only be implemented on a 66 type-1 sender [1], but provides the best possible delay performance 67 to this type of senders. The format defined in this document may 68 also be of interest to certain type-2-senders that want to exploit 69 the better bit-efficiency of this format as compared to [5]. The 70 format was designed so that it can be implemented by both type-1 and 71 type-2 receivers. 73 1.1. Specification Language 75 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 76 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 77 document are to be interpreted as described in RFC 2119. 79 2. Requirements 81 The requirements for this document are similar to those listed in 82 [5]. 84 A suspend/resume-oriented solution can provide better worst-case 85 latency than the pre-fragmenting-oriented solution defined in [5]. 86 Also, as this solution requires a new encapsulation scheme, there is 87 an opportunity to provide a slightly more efficient format. 89 Predictability, robustness, and cooperation with PPP and existing 90 hard- and firmware installations are as important with suspend/resume 91 as with pre-fragmenting. A good suspend/resume solution achieves 92 good performance even with type-2 receivers [1] and is able to work 93 with PPP hardware such as async-to-sync converters. 95 Finally, a partial non-requirement: While the format defined in this 96 draft is based on the PPP multilink protocol ([2], also abbreviated 97 as MP), operation over multiple links is in many cases not required. 99 3. General Approach 101 As in [5], the general approach is to start out from PPP multilink 102 and add multiple classes to obtain multiple levels of suspension. 103 However, in contrast to [5], more significant changes are required to 104 be able to suspend the transmission of a packet at any point and 105 inject a higher priority packet. 107 The applicability of the multilink header for suspend/resume type 108 implementations is limited, as the ``end'' bit is in the multilink 109 header, which is the wrong place for suspend/resume operation. To 110 make a big packet suspendable, it must be sent with the ``end'' bit 111 off, and (unless the packet was suspended a small number of bytes 112 before its end) an empty fragment has to be sent afterwards to 113 ``close'' the packet. The minimum overhead for sending a suspendable 114 packet thus is twice the multilink header size (six bytes, including 115 a compressed multilink protocol field) plus one PPP framing (three 116 bytes). Each suspension costs another six bytes (not counting the 117 overhead of the framing for the intervening packet). 119 Also, the existing multi-link header is relatively large; as the 120 frequency of small high-priority packets increases, the overhead 121 becomes significant. 123 The general approach of this document is to start from PPP Multilink 124 with classes and provide a number of extensions to add functionality 125 and reduce the overhead of using PPP Multilink for real-time 126 transmission. 128 This document introduces two new features: 130 1) A compact fragment format and header, and 132 2) a real-time frame format. 134 4. The Compact Fragment Format 136 This section describes an optional multilink fragment format that is 137 more optimized towards single-link operation and frequent suspension 138 (type 1 senders)/a small fragment size (type 2 senders), with 139 optional support for multiple links. 141 When operating over a single link, the Multilink sequence number is 142 used only for loss detection. Even a 12-bit sequence number clearly 143 is larger than required for this application on most kinds of links. 144 We therefore define the following compact multilink header format 145 option with a three-bit sequence number. 147 As, with a compact header, there is little need for sending packets 148 outside the the multilink, we can provide an additional compression 149 mechanism for this format: the MP protocol identifier is not sent 150 with the compact fragment header. This obviously requires prior 151 negotiation (similar to the way address and control field compression 152 are negotiated), as well as a method for avoiding the bit combination 153 0xFF, as the start of a new LCP negotiation could otherwise not be 154 reliably detected. 156 Figure 1: Compact Fragment Format 158 0 1 2 3 4 5 6 7 159 +---+---+---+---+---+---+---+---+ 160 | R | sequence | class | 1 | 161 +---+---+---+---+---+---+---+---+ 162 | data | 163 : : 164 +---+---+---+---+---+---+---+---+ 166 Having the least significant bit always be 1 helps with HDLC chips 167 that operate specially on least significant bits in HDLC addresses. 168 (Initial bytes with the least significant bit set to zero are used 169 for the extended compact fragment format, see next section.) 171 The R bit is the inverted equivalent of the B bit in the other 172 multilink fragment formats, i.e. R = 1 means that this fragment 173 resumes a packet previous fragments of which have been sent already. 175 The following trick avoids the case of a header byte of 0xFF (which 176 would mean R=1, sequence=7, and class=7): If the class field is set 177 to 7, the R bit MUST never be set to one. I.e., class 7 frames by 178 design cannot be suspended/resumed. (This is also the reason the 179 sense of the B bit is inverted to an R bit in the compact fragment 180 format -- class 7 would be useless otherwise, as a new packet could 181 never be begun.) 183 As the sequence number is not particularly useful with the class 184 field set to 7, it is used to distinguish eight more classes -- for 185 some minor additional complexity, the applicability of prefix elision 186 is significantly increased by providing more classes with possibly 187 different elided prefixes. 189 For purposes of prefix elision, the actual class number of a fragment 190 is computed as follows: 192 - If the class field is 0 to 6, the class number is 0 to 6, 194 - if the class field is 7 and the sequence field is 0 to 7, the 195 class number is 7 to 14. 197 As a result of this scheme, the classes 0 to 6 can be used for 198 suspendable packets, and classes 7 to 14 (where the class field is 7 199 and the R bit must always be off) can be used for non-suspendable 200 high-priority classes, e.g., eight highly compressed voice streams. 202 5. The Extended Compact Fragment Format 204 For operation over multiple links, a three-bit sequence number will 205 rarely be sufficient. Therefore, we define an optional extended 206 compact fragment format. The option, when negotiated, allows both 207 the basic compact fragment format and the extended compact fragment 208 format to be used; each fragment indicates which format it is in. 210 Figure 1: Extended Compact Fragment Format 212 0 1 2 3 4 5 6 7 213 +---+---+---+---+---+---+---+---+ 214 | R | seq LSB | class | 0 | 215 +---+---+---+---+---+---+---+---+ 216 | sequence -- MSB | 1 | 217 +---+---+---+---+---+---+---+---+ 218 | data | 219 : : 220 +---+---+---+---+---+---+---+---+ 222 In the extended compact fragment format, the sequence number is 223 composed of three least significant bits from the first octet of the 224 fragment header and seven most significant bits from the second 225 octet. (Again, the least significant bit of the second octet is 226 always set to one for compatibility with certain HDLC chips.) 228 For prefix elision purposes, fragments with a class field of 7 can 229 use the basic format to indicate classes 7 to 14 and the extended 230 format to indicate classes 7 to 1030. Different classes may use 231 different formats concurrently without problems. (This allows some 232 classes to be spread over a multi-link and other classes to be 233 confined to a single link with greater efficiency.) For class fields 234 0 to 6, i.e. suspendable classes, one of the two compact fragment 235 formats SHOULD be used consistently within each class. 237 If the use of the extended compact fragment format has been 238 negotiated, receivers MAY keep 10-bit sequence numbers for all 239 classes to facilitate senders switching formats in a class. When a 240 sender starts sending basic format fragments in a class that was 241 using extended format fragments, the 3-bit sequence number can be 242 taken as a modulo-8 version of the 10-bit sequence number, and no 243 discontinuity need result. In the inverse case, if a 10-bit sequence 244 number has been kept throughout and no major slips of the sequence 245 number have occurred, no discontinuity will result, although this 246 cannot be guaranteed in the presence of errors. (Discontinuity, in 247 this context, means that a receiver has to resynchronize sequence 248 numbers by discarding fragments until a fragment with R=0 has been 249 seen.) 251 6. Real-Time Frame Format 253 This section defines how fragments with compact fragment headers are 254 mapped into real-time frames. This format has been designed to 255 retain the overall HDLC based format of frames, so that existing 256 synchronous HDLC chips and async to sync converters can be used on 257 the link. Note that if the design could be optimized for async only 258 operation, more design alternatives would be available [4]; with the 259 advent of V.80 style modems, asynchronous communications is likely to 260 decrease in importance, though. 262 The compact fragment format provides a compact rendition of the PPP 263 multilink header with classes and a reduced sequence number space. 264 However, it does not encode the E-bit of the PPP multilink header, 265 which indicates whether the fragment at hand is the last fragment of 266 a packet. 268 For a solution where packets can be suspended at any point in time, 269 the E-bit needs to be encoded near the end of each fragment. The 270 real-time frame format, to ensure maximum compatibility with type 2 271 receivers, encodes the E-bit in the following way: Any normal frame 272 ending also ends the current fragment with E implicitly set to one. 273 This ensures that packets that are ready for delivery to the upper 274 layers immediately trigger a receive interrupt even at type-2 275 receivers. 277 Fragments of packets that are to be suspended are terminated within 278 the HDLC frame by a special ``fragment suspend escape'' byte (FSE). 279 The overall structure of the HDLC frame does not change; the 280 detection and handling of FSE bytes is done at a layer above HDLC 281 framing. 283 The suspend/resume format with FSE detection is an alternative to 284 address/control field compression (ACFC, LCP option 8). It does not 285 apply to frames that start with 0xFF, the standard PPP-in-HDLC 286 address field; these frames are handled as defined in [6] and [7]. 287 (This provision ensures that attempts to renegotiate LCP do not cause 288 ambiguities.) 290 For frames that do not start with 0xFF, suspend/resume processing 291 performs a scan of every HDLC frame received. The FCS of the HDLC 292 frame is checked and stripped. Compact fragment format headers (both 293 basic and extended) are handled without further FSE processing*. 294 Within the remaining bytes of the HDLC frame (``data part''), an FSE 295 byte is used to indicate the end of the current fragment, with an E 296 bit implicitly cleared. All fragments up to the last FSE are 297 considered suspended (E = 0); the final fragment is terminated (E = 298 1), or, if it is empty, ignored (i.e., the data part of an HDLC frame 299 can end in an FSE to indicate that the last fragment has E = 0). 301 Each fragment begins with a normal header, so the structure of a 302 _________________________ 303 * As the FSE byte was chosen such that it never occurs in com- 304 pact fragment format headers, this does not require any specific 305 code. 307 frame could be: 309 Figure 2: Example frame with FSE delimiter 311 0 1 2 3 4 5 6 7 312 +---+---+---+---+---+---+---+---+ 313 | R | sequence | class | 1 | 314 +---+---+---+---+---+---+---+---+ 315 | data | 316 : : 317 +---+---+---+---+---+---+---+---+ 318 + FSE + previous fragment implicitly E = 0 319 +---+---+---+---+---+---+---+---+ 320 | R | sequence | class | 1 | 321 +---+---+---+---+---+---+---+---+ 322 | data | 323 : : 324 +---+---+---+---+---+---+---+---+ 325 | Frame | previous fragment implicitly E = 1 326 | CRC | 327 +---+---+---+---+---+---+---+---+ 329 The value chosen for FSE is 0xDE (this is a relatively unlikely byte 330 to occur in today's data streams, it does not trigger octet stuffing 331 and triggers bit stuffing only for 1/8 of the possible preceding 332 bytes). 334 The remaining problem is that of data transparency. In the scheme 335 described so far, an FSE is always followed by a compact fragment 336 header. In these headers, the combination of a class field set to 7 337 with R=1 is reserved. Data transparency is achieved by making the 338 occurrence of an FSE byte followed by one of 0x8F, 0x9F, ... to 0xFF 339 special. 341 Figure 3: Data transparency with FSE bytes present 343 0 1 2 3 4 5 6 7 344 +---+---+---+---+---+---+---+---+ 345 | R | sequence | class | 1 | 346 +---+---+---+---+---+---+---+---+ 347 | data | 348 : : 349 +---+---+---+---+---+---+---+---+ 350 + FSE + fragment NOT terminated 351 +---+---+---+---+---+---+---+---+ 352 | R | S | T | U | 1 | 1 | 1 | 1 | R always is 1 353 +---+---+---+---+---+---+---+---+ 354 | data | fragment continues 355 : : 357 In a combination of FSE/0xnF (where n is the first four-bit field in 358 the second byte, RSTU in Figure 3), the n field gives a sequence of 359 four bits indicating where in the received data stream FSE bytes, 360 which cannot simply be transmitted in the data stream, are to be 361 added by the receiver: 363 0x8F: insert one FSE, back to data 364 0x9F: insert one FSE, copy two data bytes, insert one FSE, back to data 365 0xAF: insert one FSE, copy one data byte, insert one FSE, back to data 366 0xBF: insert one FSE, copy one data byte, insert two FSE bytes, back to data 367 0xCF: insert two FSE bytes, back to data 368 0xDF: insert two FSE bytes, copy one data byte, insert one FSE, back to data 369 0xEF: insert three FSE bytes, back to data 370 0xFF: insert four FSE bytes, back to data 372 The data bytes following the FSE/0xnF combinations and corresponding 373 to the zero bits in the N field may not be FSE bytes. 375 This scheme limits the worst case expansion factor by FSE processing 376 to about 25 %. Also, it is designed such that a single data stream 377 can either trigger worst-case expansion by octet stuffing (or by bit 378 stuffing) or worst-case FSE processing, but never both. Figure 4 379 illustrates the scheme in a few examples; FSE/0xnF pairs are written 380 in lower case. 382 Figure 4: Data transparency examples 384 Data stream FSE-stuffed stream 386 DD DE DF E0 DD de 8f DF E0 387 01 DE 02 DE 03 01 de af 02 03 388 DE DA DE DE DB de bf DA DB 390 In summary, the real-time frame format is a HDLC-like frame delimited 391 by flags and containing a final FCS as defined in [7], but without 392 address and control fields, containing as data a sequence of FSE- 393 stuffed fragments in compact fragment format, delimited by FSE bytes. 394 As a special case, the final FSE may occur as the last byte of the 395 data content (i.e. immediately before the FCS bytes) of the HDLC-like 396 frame, to indicate that the last fragment in the frame is suspended 397 and no final fragment is in the frame (e.g., because the desirable 398 maximum size of the frame has been reached). 400 7. Implementation notes 402 7.1. MRU Issues 404 The LCP parameter MRU defines the maximum size of the packets sent on 405 the link. Async-to-sync converters that are monitoring the LCP 406 negotiations on the link may interpret the MRU value as the maximum 407 HDLC frame size to be expected. 409 Implementations of this specification should preferably negotiate a 410 sufficiently large MRU to cover the worst-case 25 % increase in frame 411 size plus the increase caused by suspended fragments. If that is not 412 possible, the HDLC frame size should be limited by monitoring the 413 HDLC frame sizes and possibly suspending the current fragment by 414 sending an FSE with an empty final fragment (FSE immediately followed 415 by the end of the information field, i.e. by CRC bytes and a flag) to 416 be able to continue in a new HDLC frame. This strategy also helps 417 minimizing the impact of lengthening the HDLC frame on the safety of 418 the 16-bit FCS at the end of the HDLC frame. 420 7.2. Implementing octet-stuffing and FSE processing in one automaton 422 The simplest way to add real-time framing to an implementation may be 423 to perform HDLC processing as usual and then, on the result, to 424 perform FSE processing. A more advanced implementation may want to 425 combine the two levels of escape character processing. Note, 426 however, that FSE processing needs to wait until two bytes from the 427 HDLC frame are available and followed by a third to ensure that the 428 bytes are not the final HDLC FCS bytes, which are not subject to FSE 429 processing. I.e., on the reception of normal data byte, look for an 430 FSE in the second-to-previous byte, and, on the reception of a frame- 431 end, look for an FSE as the last data byte. 433 8. Negotiable options 435 The following options are already defined by MP [2]: 437 o Multilink Maximum Received Reconstructed Unit 439 o Multilink Short Sequence Number Header Format 441 o Endpoint Discriminator 443 The following options are already defined by MCML [5]: 445 o Multilink Header Format 447 o Prefix Elision 449 This document defines two new code points for the Multilink Header 450 Format option. 452 8.1. Multilink header format option 454 The multilink header format option is defined in [5]. A summary of 455 the Multilink Header Format Option format is shown below. The fields 456 are transmitted from left to right. 458 Figure 5: Multilink header format option 460 0 1 2 3 461 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 463 | Type = 27 | Length = 4 | Code | # Susp Clses | 464 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 466 As defined in [5], this LCP option advises the peer that the 467 implementation wishes to receive fragments with a format given by the 468 code number, with the maximum number of suspendable classes (see 469 below) given. This specification defines two additional values for 470 Code, in addition to those defined in [5]: 472 - Code = 11: basic and extended compact real-time fragment format 473 with classes, in FSE-encoded HDLC frame 475 - Code = 15: basic compact real-time fragment format with classes, 476 in FSE-encoded HDLC frame 478 An implementation MUST NOT request a combination of both LCP Address- 479 and-Control-Field-Compression (ACFC) and the code values 11 or 15 for 480 this option. 482 The number of suspendable classes negotiated for the compact real- 483 time fragment format only limits the use of class numbers that allow 484 suspending. As class numbers of 7 and higher do not require 485 additional reassembly space, they are not subject to the class number 486 limit negotiated. 488 9. References 490 [1] C. Bormann, ``Providing integrated services over low-bitrate 491 links'', Work in Progress (draft-ietf-issll-isslow-05.txt), 492 April 1999. 494 [2] K. Sklower, B. Lloyd, G. McGregor, D. Carr, T. Coradetti, ``The 495 PPP Multilink Protocol (MP)'', RFC 1990, August 1996 (obsoletes 496 RFC1717). 498 [3] W. Simpson, ``PPP in Frame Relay'', RFC 1973, June 1996. 500 [4] R. Andrades, F. Burg, ``QOSPPP Framing Extensions to PPP'', Work 501 in Progress (draft-andrades-framing-ext-00.txt), September 1996. 503 [5] C. Bormann, ``The Multi-Class Extension to Multi-Link PPP'', 504 Work in Progress (draft-ietf-issll-isslow-mcml-05.txt), April 505 1999. 507 [6] W. Simpson, Editor. ``The Point-to-Point Protocol (PPP)'', RFC 508 1661, July 1994 (Obsoletes RFC1548, also STD0051). 510 [7] W. Simpson, Editor. ``PPP in HDLC-like Framing'', RFC 1662, July 511 1994 (Obsoletes RFC1549, also STD0051) 513 10. Author's address 515 Carsten Bormann 516 Universitaet Bremen FB3 TZI 517 Postfach 330440 518 D-28334 Bremen, GERMANY 519 cabo@tzi.org 520 phone +49.421.218-7024 521 fax +49.421.218-7000 523 Acknowledgements 525 The participants in a lunch BOF at the Montreal IETF 1996 gave useful 526 input on the design tradeoffs in various environments. Richard 527 Andrades, Fred Burg, and Murali Aravamudan insisted that there should 528 be a suspend/resume solution in addition to the pre-fragmenting one 529 defined in [5]. The members of the ISSLL subgroup on low bitrate 530 links (ISSLOW) have helped in coming up with a set of requirements 531 that shaped this solution.