2.4.7 Internet Traffic Engineering (tewg)

NOTE: This charter is a snapshot of the 49th IETF Meeting in San Diego, California. It may now be out-of-date. Last Modified: 28-Nov-00


Ed Kern <ejk@tech.org>
Jim Boyle <jim.boyle@level3.com>

Operations and Management Area Director(s):

Randy Bush <randy@psg.com>
Bert Wijnen <bwijnen@lucent.com>

Operations and Management Area Advisor:

Randy Bush <randy@psg.com>

Technical Advisor(s):

Rob Coltun <rcoltun@redback.com>

Mailing Lists:

General Discussion:te-wg@uu.net
To Subscribe: te-wg-request@uu.net
In Body: subscribe
Archive: ftp://ftpext.eng.us.uu.net/tewg

Description of Working Group:

Internet Traffic Engineering is that aspect of network engineering which is concerned with the design, provisioning, and tuning of operational internet networks. It applies business goals, technology and scientific principles to the measurement, modeling, characterization, and control of internet traffic, and the application of such knowledge and techniques to achieve specific service and performance objectives, including the reliable and expeditious movement of traffic through the network, the efficient utilization of network resources, and the planning of network capacity.

The Internet Traffic Engineering Working Group (tewg) defines, develops, specifies, and recommens principles, techniques, and mechanisms for traffic engineering. The working group will also serve as a general forum for discussing possible improvements to IETF protocols to advance the traffic engineering function.

The primary focus of the tewg is the measurement and control aspects of intra-domain internet traffic engineering. This includes provisioning, measurement and control of intra-domain routing, and measurement and control aspects of intra-domain network resource allocation. Techniques already in use or in advanced development for traffic engineering include ATM and Frame Relay overlay models, MPLS based approaches, constraint-based routing, and traffic engineering methodologies in Diffserv environments. The tewg describes and characterizes these and other techniques, documents how they fit together, and identifies scenarios in which they are useful.

The working group may also consider the problems of traffic engineering across autonomous systems boundaries.

The tewg interacts with the control and measurement plane working groups to abstract and define those parameters, measurements, and controls that traffic engineering needs in order to engineer the network.

TEWG also interacts with other groups whose scopes intersect. These include various working groups in the routing area (e.g. mpls, is-is, ospf, etc), the transport area (e.g. diffserv, ippm, rap, rtfm,), and the operations and management area (e.g. policy, rmonmib, disman, etc).

Goals and Milestones:

Nov 00


Solicit TEBCP drafts concerning requirements, approaches, lessons learned from use (or non use) of TE techniques in operational provider environments.

Nov 00


Solicit vendor implementation similarities and differences informational draft (TEIMP)

Nov 00


Review and comment on operational TEMIB

Dec 00


Update operational TE MIB draft

Dec 00


TEBCPs submitted for WG comment

Jan 01


Comments to TEBCP authors for clarifications

Jan 01


First draft of TEIMP

Jan 01


First draft of TEM

Feb 01


Another update of operational TEMIB draft

Mar 01


Submit revised TEBCPs and REAPP to AD/IESG for review

Apr 01


Progress operational TE MIB to AD review

May 01


TEIMP draft to AD review

Jun 01


Submit TEM draft for AD review

Jun 01




No Request For Comments

Current Meeting Report

Internet Traffic Engineering (TEWG)
Tuesday 9-11:30, December 12, 2000

45 minutes - new charter, scoping, work items, contributions needed, Qs

15 minutes - Kireeti - mib update, changes, timeframe

15 minutes - Francois - diffserv TE, wg item time?

15 minutes - Senthil - Multi-area TE

15 minutes - Kireeti - Multi-area TE

10 minutes - Marcus - Policy Framework

10 minutes - Haskin - Fast Reroute Mechanism

10 minutes - wrapup

Also see associated agendas in the following WGs: ISIS, OSPF, MPLS, IPO, COMA

Thanks to Giles Heron and Vijay Gill who took notes for minutes.
Minutes of the TEWG session 12/12/00

Jim B. called the meeting to order, and indicated that the group had been refocussed over the last couple of IETFs. Back in june or july, started to refocus the working group on what people are doing today in production networks. This resulted in a needed revised and focussed charter.

Ed took the group through the new charter, and stated that the main focus of the meeting would be the new charter. He checked how many people had read the new charter and drafts.

In ABQ and Pittsburgh, sat down and decided short term goals Document requirements from operators How to solve the specific requirements that operators outlined

Ed indicated that there were issues in getting input from SPs - since lack of perceived need of TE or feel their techniques are too gory to publicize

The WG had also become a home for drafts with no home.

Jim commented that information was being solicited in the form of BCP drafts from SP. Multivendor interoperability of TE implementations is a particular issue. Some success had been had with the TE MIB submitted by Kireeti Kompella. The focus of the charter right now is tactical. There is a framework draft with the IESG, and the group wants to push forward to IESG last call.

Questions on charter

Bora Akyol - Does the charter explicitly includes or excludes non-MPLS traffic engineering approach such as changing metrics.

Jim B: Perfectly acceptable, need more best current practices Focus is on MPLS TE, but there is room to show different ways to optimize network resources, such as manipulation of IGP or BGP. Ed further commented that the group would always consider other options to MPLS.

Kireeti: TE for optical networks?

Jim responded that his personal opinion was that approaches that were suitable for IP networks, but were also useful for optical networks were probably good candidates for being considered general and addressed in another group.

Randy commented that as discussed at CCAMP, this is the IETF I for Internet and IP. We are not trying to get telephone companies to configure their networks. Draw a line between IP being used as the control plane for various kinds of traffic engineering. Our expertise is protocols for dealing with IP. Using TE for generic optical networks unless it is carrying IP, is not within our scope

Jim B: who knows what is in those circuits. People need to develop IP control protocols for these particular applications. IP optical networks aren't the purview of this WG, though.

Fred Baker seconded Randy's comment and stated that our focus is on making the Internet work. If these techniques are useful for other purposes then that is OK, but not our focus.

Vishal: Question about scope of restoration work, couple of IETF ago, was decided to be outside scope of TE, are now again in scope of TEWG.

Randy: Lets step back to 15 thou meters up. TEWG and PPVPN and other needs like that need to be able to look at the conditions of a network, change the conditions of the networks, CCAMP is trying to abstract from the layers beneath (frame, MPLS, ATM) what general ways we can talk about those things generically instead of specially, so that TEWG doesn't have to reach down and know the difference between optical etc.

Choose to buy a circuit from NYC to London on Yellow (no diversity) and back it up with a satellite circuit (ATM) I may not want to know about actual physical transports, need only characteristics such as bandwidth and delay and use them for policy. CCAMPs job is to abstract that. OS viewpoint mechanism and policy to make use of those mechanisms (need separation)

Tell me what things I can fall over to within 50 ms, which are cheap etc.

Clarification Q. Are you saying that the mechanisms for restoration are not in scope?

Jim responded that the requirements are driven out of TEWG, but into CCAMP (along with other WG's requirements.) CCAMP compiles those and pushes them down, possibly into protocol WG.

Fred commented that 10 minutes of the hour had been used up discussing what was in scope.

Randy reponded that there was 45 minutes allocated to discussion. He commented that the way CCAMP talks to other protocols is an API which will need to be developed. The group wants to avoid reaching all the way down to optical and developing the protocols.

The original questioner indicated that if these protocols don't have a home here then why are they being discussed.

Jim restated that the requirements were what were being discussed, rather than the protocols.

Ed asked for some flexibility, given that the charter was sent out before the "great reorginization" happened.

Jim stated that as Randy had mentioned this group can prototype protocols, but is not developing them.

Jim indicated that there were maybe 2 more issues to discuss. 1 the group needed to discuss issues rather than have presentations. 2 was consensus. This is the majority of the WG agreeing.

Fred stated that a charter is an agreement between the WG and the IESG.

Randy restated the process.

Jim asked for a show of hands of those who had:
1.Read the charter. (25% of room)
2.Agreed with it (25% of room)
3.Had issues (1 noted below)

Scott Hahn: One issue was brought up (which documents should be in this WG) and will be addressed afterwards. Randy commented that sorting 134 documents into a taxonomy of WG was not a trivial exercise.

TEWG Mib: Kireeti

Several people commented on the mailing list some privately, thanks for feedback. Thanks to Keith for the very detailed comments

Some changes discussed, not yet resolved:

1. Should it be read only or read-modify. Keith had indicated that you can't just change this. Kireeti's personal pref is that it should be read-only as is operational MIB. Kireeti said he comes from a school that says you don't configure via SNMP. Has had feedback from carriers and they seem happy with that.

2. Index for tunnel info. Currently a name, but some SNMP stacks don't like this. Not practical - so need to change it.

3. Keith had indicated that admin groups needed to have a "converged" name. These are link properties. Need a mapping between ifindices and admin groups. This also impacts some other changes.

Some new and old questions include:

1. Do we want to carry TE info (link resources) in the TE MIB? Do we want a TE link state database in the MIB? One reason for this is that ISIS and OSPF have this info, but not in MIB (as is new.) Do it once in TE or once for each protocol. (3 times in fact because of OSPFv3.)

2. Do we want generalized tunnel info (GMPLS.) There's a bunch of stuff. Do we even go there, do we wait for CCAMP, or do we go ahead here.

Comments on presentation:

Alex: As far as the read-only and read/modify, I would go with read-only. As far as providing the information of the information in the IGP, we should make it TE specific vs. IGP specific and GMPLS: we should just move forward.

Jim B: any questions? Send to mailing list, another version of draft this month but it'll probably be in January (holiday season). Ask questions on mailing list re: kireetis talk.

April 1st milestone to IESG

Francois - DiffServ aware TE

Jim stated that Francois had done a good job of pointing out where the current techniques fall short. Asked if the chairs could have feedback on whether this should be adopted as a WG item.

Francois presented the draft.

2 drafts had been split into 4 drafts since Pittsburgh. At this session would just discuss the requirements doc. The other 3 (OSPF, ISIS, RSVP.)

Aim is to discuss what is required - can't be met by existing protocols. Then go on to discuss the proposals.

Currently can run MPLS TE and DiffServ together. Achieves CBR for aggregate bandwidth constraint - don't consider what is being transported. May be sufficient for most environments. What isn't being covered?

DiffServ principle - good EF behaviour requires that EF traffic less than "reasonable" % of link capacity. BE behaviour fine if aggregate load is 100% of link. An example was presented showing that a link might have capacity for 10mbps of BE but not for 10mbps of EF.

Would like CBR to be able to include a link for BE, and exclude it for voice/EF. Current TE has a single unreserved BW value -> link is either included or excluded for all CoS.

Where is this useful?
1. Distributed route computation.
2. Voice traffic is significant compared to link speed.
3. Distribution of traffic across classes is not consistent everywhere.

Some customers have these requirements?
1.Global SPs where have constrained transoceanic links with significant % of voice.
2.Major telcos who want to trunk their voice over the IP network.

At least one vendor has an implementation resembling what is spelled out in the draft. Some service providers have tested these implementations (according to the vendor).

What are the principles of the implementation?

1. Group many CoS into a few Class Types (CT) e.g one CT for BE, one for AF1/AF2, one for EF.
2. May want to limit different CT to different rates.
3. Need IGP to advertise unreserved B/W per CT.
4. CBR can use value for relevant CT.
5. Signalling protocol can set up for relevant CT, to get proper admission control.


1. How many CT do we need? Francois's feeling is that most urgent case is separating voice/data (so 2 CT.) But also may want to split out AF. So 4 CT is probably about right. This supports 6 or 8 types of traffic. Some may want one CT (existing TE.) Some 2 (voice + data), some 3 (voice, low-loss data, BE.) Personally says should specify what we're sure we need. Jim suggested that we should be able to do more, or less, it should be flexible, to avoid changing later on.

2. How does preemption work? It's not clear as to if one CT should always pre-empt another. Rather than a fixed relationship the solution is to have no constraints at all. 8 levels of preemption. Preemption could work within/across CT. Suggestion is that it will be completely independent, so goes across CT. Up to the SP to define this. Examples were given.

3. How to minimise IGP scalability impact? Feedback is that many SPs don't use all 8 preemption levels. Solution is to only advertise the used values. This has changed since last proposal. Last proposal required SP to configure on router how many levels were used + configure on router. That's a pain (config + changes.) New scheme is that the IGP simply removes the redundant B/W values, except for P0. The existing 8 values have to be advertised for backwards compatibility. Only a few additional ones are likely to be added (examples given.)

4. How do we compute unreserved b/w per CT? One simple model proposed. Example given. Potential enhancements discussed in the draft. Not yet sure if this will work.

Proposal. Should TEWG take ownership of the requirements doc.? This is currently owned by the MPLS WG, and will be progressed there until a decision is made.

Jim opened the floor for questions.

Vijay stated that from an operational perspective this was adding significant complexity. Other options were possible. For example expanding the reservable b/w on a link if there was signficant amount of voice on it.

Jim replied that there are engineering techniques that can handle this, but this is more generic.

Will this change the TE MIB?

Kireeti stated that if the MIB isn't doing the LS stuff then it won't change the MIB. If the MIB is doing the IGP stuff then per class-type B/W reservation would have to be included. Kireeti addressed 2 issues. He likes the idea the B/W at P0 > B/W at P1. Think carefully before throwing away. Thinks that oversubscription has not been addressed here. Problem is you can't subtract B/W any more. Will work fine if don't have much oversubscription.

Is aim of draft to reduce IGP info?

No - aim is to allow CBR on different types of traffic, but while mimimising the increase of IGP data.

Why not advertise B/W by class rather than CT?

Goal of mimimising impact on IGP. By grouping classes together you can achieve this. No need to, for example, advertise AF1 and AF2 separately. Scheduling remains on a class basis.

Angela chiu: Cannot put more than 50% of EF traffic on link. ATT tried to engineer VOIP, found out that you would get more than 50% capacity of the link. This motivates two classes.

Comment on how to make oversubscription work. If advertise peak + avg rate when establishing the LSP can use avg for link b/w used.

Francois stated that with this model can do oversubscription at the link level. For example can make 60% of link reservable for voice when actually only allow 50%. So can build overbooking ratio in.

Kireeti responed that if the oversubscription ratios for the different traffic types were different couldn't just subtract them.

Comment that admission of micro-flows into LSPs could be used for oversubscription rather than at this level. This makes more sense as may want to have different policies for the different CT.

Jim asked for a feel on this from the WG. 4 Q's, want a show of hands.

1. Is the WG interested in pursuing work of this type?

2. Is the proposal in the right direction?

3. Should we adopt the requirements doc in this WG?

4. Should we adopt the protocol docs in this WG?

Jim suggested that non affiliated providers should also add their requirements to the requirements doc. Send comments on requirements + protocols to the WG list. Let's not work on this, and then say in a year's time that the solution doesn't meet our requirements.

Multi-Area TE. 2 Presentations


Basic problem - how to set up TE LSPs across IGP areas. Primary problem is that the TE data is only flooded intra-area. Need to exend this. Other problem is how to get the signalling components to carry the info across areas.

Solution needs to:
1. conform to existing standards
2. be backwards compatible
3. be scalable

Stitching isn't scalable.

Summary LSAs defined to carry info out of area while reducing table size. If allow TE info in summaries will get info out of area + reduce crankback.

Proposal is to have primary + secondary constraints. If Primary fails, secondary may succeed.

1. Extended IGP
2. Have extended CSPF
3. signalling extensions

Example given:

TE info is propagated from area 2 into backbone area. Propagated to destination area. Know if LSP can be setup. Also know which ABR is closest + has best possibiliy of setting it up. Based on that can set up LSP. LSP is in sections.

Several different metrics.

1. Additive metric (as used in OSPF/ISIS today.)
2. Max/min metric.
3. Colors, combinations of colors

Signalling extended to allow crankback using primary/secondary signalling. If exhaust all possibilities crankback to singalling LSR and try again. Secondary reduces the amount of crankback, since can use secondary criterion if primary can't be met.

Conclusion. Have extended IGP, CSPF, signalling. Can find optimal path using TE info. If not all available can find a path which may not be optical. TE feedback can be used when doing crankback to try alternate routes. Uses existing signalling models, so nothing really new. No new h/w required. Is a scalable approach. Allows dynamic setup which is distributed across the ABRs in the domain - rather than centralised computation, which isn;'t very scalable.

Multi-Area TE - Kireeti

Recapped issue of TE being single-area only, plus the requirement for multi-area networks for scalability and administrative separation (e.g. For optical networks with routed edge - optical guys don't want to be impacted by router flaps.

TE DB in ISIS/OSPF exchanges info in each area. Question is how do I do an inter-area LSP?

Many approaches to per-area path computation.

1. Per Area Path computation
2. Delegate computation to ABR.
3. Request TE info and do at source
4. Forwarding adjacencies.

Per Area. A signals path to X (ABR.) A tells X "these are my constraints." X then knows that he needs to get to Z (ABR for tail-end area.) X computes path and tells Z the constraints. Z computes a path to B. What if can't find a path. Crankback. The worst case is n*n retries to compute a path. May only be n + n since if Z can't get to be doesn't really matter if can get there from X or Y.

Delegate. Example A delegates to Y. If crankbacks will crankback all the way to A. As many retries as tail-end ABRs.

Request info. Example A requests info from Y. Y requests from Z. Therefore A can do computation itself.

Forwarding Adjacencies.

What mechanisms do we need?

1. Means of communicating constraints.
2. Means of asking ABR for a path, plus returning ERO
3. Means of asking ofr topology information.
4. ?

Information required. ? Multiple approaches, one size doesn't fit all.

Doesn't think summarising TE info + metrics works. Not a single constraint.

Question. Assertion that we need this. Is this true? Why not use hierarchical LSPs? That's the whole point of a backbone (scalability.) This seems to be rather like PNNI.

Jim asked if any SP was hitting a problem with this. No hands. He stated that he feels it is an issue with the optical domains.

Alex? Bora?. This doesn't address the number of labels maintained in the backbone. Need to use tunnelling. If you accept that can be optical switches will accept that this is necessary. How many times will ABRs need to calculate CSPF. Also a constraint. Would also like to see a simulation of this. The summarization may actually increase the number of crankbacks. Issue for Kireeti. Why flood topology of backbone area to non-backbone areas. Only head-end LSRs need to know, internal routers don't. Thinks it will be a mixture of approaches - not a single approach. Delegated CSPF calculation + announcing forwarding adjacecencies resulting from this may be the best approach.

George S. stated that at least 15 SP have expressed interest in this. Don't need it today, but want it on the roadmap. So we need a solution to this soon. 2nd statement was that this is control not data - so applies to optical etc. If he understands what IESG is doing then believes that this lives in CCAMP.

Kireeti responded to Bora/Alex. One could compute a path from A to X to Z to B. Having computed it can trigger the auto-generation of a fwdg adjacency in the bbone.

Jim stated that George's point is well taken. Please think about the requirements, especially if an SP. SPs need to step forward with requirements, rather than having software vendors "acting in the name of the unnamed".

Policy Framework - Ritu Chadha

Ritu: Was originally 2 drafts presented in Pittsburgh at PF WG. Combined into one draft.

Policy Based management of MPLS for QoS + TE. Network wide management rather than NE management + individual trunks/LSPs. Want to be independent of signalling protocol + management protocol. Aim is to have high-level support to manage networks.

Example of what they want to do. SP has a large n/w and some policy rules to govern its behavior. Rules are condition+action. 1 is 5pm/weekday -> reduce b/w by 50%. Some customers may only need "gold" during business hours, so reduce gold trunks by 50% at 5pm. Indicates the entity to which the rule applies. This applies to all gold trunks. Simiar rule for 8am weekdays to restore the b/w to 100%. Each traffic trunk has one profile associated with it. E.g. 5p.m. Monday. The b/w for gold trunks is reduced 50%. If can apply this to any number of trunks it allows the SP to easily manage network by setting a number of policies.

Markus continued by describing the model. This is a rule-based specification. Fits with Francois' presentation. IF this is true THEN do something. More examples:

1. Working hours
2. New SLA -> new LSPs required
3. Rerouting round ? LSPs.

Open issues.

1. What conditions are needed
2. What actions are needed
3. How to define reverse direction of trunk
4. Is this a working item for this WG?

Jim thinks 4 is an open Q to leave to the TEWG list. Show of hands says no consensus, plus not within the charter. Mention of Steven Wrights work which explicitely points to ways to control load on equal cost LSPs, as well as the more general policy issues.

Randy stated that PF WG has some immediate targets to validate their tech. This isn't one of them.

Fast Reroute - Dimitri

Jim - this has been implemented. We need to be thinking about this area - particularly wrt our requirements document.

Dimitri - This draft was published a while ago. Believes it is now in its final form + complete.


1. Mimimise the number of alternative paths required to protect the active end-to-end path
2. Provide very quick protection
3. Provide local restoration/repair
4. Ecomomical use of protection resources

Protection path follows active path (in reverse.) When failure happens in the middle the switchover will occur at the point of failure. Comparible to SONET (sub 50msec.) Traffic flows back to source and is routed to destination?

Proposal also has provision for shortcuts. May have unacceptible round-trip-delay. Shortcut can be from middle point so if failure happens closer in path can shortcut to destination and get acceptible end-to-end delay.

Can protect multiple LSPs between one pair of LSPs. Example shown.

Aim is to publish as an informational RFC. Would be nice to find a home for the draft. Initial intention is to propose in this WG.

Jim commented that the WG needs to think about restoration. Applicability also in the optical case, so not sure where this work should be hosted. Extensive info in the framework document. Make sure we cover providers' requirements.

Dimitri - Emphasis that this is just one technique.

Jim agreed, and stated that there hasn't been much co-ordination between the different techniques. Important issue is that requirements are stated.

Inter-area MPLS path setup - Chang-Yin Lee - Nortel

Issues, solution, optimization.

Problem. How do we do a contrained path from X to Y where X,Y in different areas. As already stated in other talks X does not have full info to do this.

E.g. X wants 2 paths X to Y which are diverse.

Can we use the ABRs to do this? X could request the info from ABR.

Observation. Only the ABRs need to know the TE link states. ABRs are "Traffic Exchangers". No need to flood the link states.

So once Traffic Exchanger gets the TE link state will send it to other Traffic Exchangers, and not to all other routers.

So to set up a path from X to Y, X asks its local Traffic Exchanger.

This enables inter-area path setup and reduces crankbacks.

Relationship to other drafts: [** chair note, this is more valued than reviews of presented approach, as all should have read drafts - jim, ed]

If the summary TE LSA is available the Traffic Exchangers can use this to select the optimal path.

Similar to Kireeti's draft (query to Traffic Exchanger.) Difference is that Traffic Exchanger reduces flooding. [chair note: it's similar to one of the many proposals in Kireeti's draft]

Alex - flooding belongs in IGP workgroups (OSPF and ISIS.) So need to bring this draft to their notice. Discussion on this offline.

Ed wrapped up.


Diff-Serv-aware Traffic Engineering
Inter-area MPLS TE Architecture and Protocol Extensions
Policy Framework MPLS Information Model for QoS and TE
A Method for Setting an Alternative Label Switched Path to Handle fast Reroute
Inter-area MPLS Path Setup - Some Issues and Suggestions