closed.New IETF Standards Track Discussion BOF (newtrk)


Thursday, November 13 at 1530-1730


CHAIR: Scott Bradner <sob@harvard.edu>


Minutes taken by Spencer Dawkins


1/ Scott presented a description of current IETF Standards Track (see 
presentation)


Scott described the history of the IETF standards process, which has had 
three formal stages since RFC 1083 by Jon Postel.  RFC 1083 had 
"proposed protocols" as the first stage. Scott did not know why it became 
"proposed standard" in late 1990 or early 1991. RFC 1310 was the first 
standalone process description before RFC 1310 the description was part of 
the IANA standards report.


Scott noted that some documents have many more informal stages - for 
example the current BGP draft is version 22.


The requirements for Draft Standard include multiple interoperable 
implementations to demonstrate document clarity and a requirement for each 
feature to be implemented to eliminate unused features.  Scott reminded us 
that Draft Standard was changed in RFC 2026 to include a test for IPR 
licensing.


Full Standard is Draft Standard with wide spread use.


2/ observations from problem working group - Elwyn Davies (See 
presentation)


Elwyn described the work of the Problem Working Group relating to the IETF 
standards track


He quoted from 2.4 section of problem statement draft, rev 5 and noted that 
some people felt that there were some issues relating to the currently 
implemented IETF standards process.


        "three-stage process not used properly" - progression was rare
        higher quality bar for Proposed Standard (at least a 
perception)
        quality checked by thought exercises, not running code
        Proposed Standards deployed as if they were mature
        this, in turn, raises the quality bar and slows things down,
        so people sometimes deploy I-Ds instead of waiting for Proposed 
Standards


There is no IETF aftercare for standards including no systematic way to 
receive bug reports and no periodic reviews


Scott Brim: a purpose of Draft Standard is a quality/clarity check of the 
documentation


Bob Hinden specifically, that one can implement from the spec.  
basically we've lost our running code heritage


Dol strom: vendors don't wait for proposed standard


3/ what other SDOs do - Scott Bradner


Scott: W3C: W3C uses three-plus stages, pretty much what we use working 
draft is internal-only (See presentation)


Joel Halpern: ISO: ISO process stars with a committee draft with number 
assigned at the beginning of the process, the formal stages include Draft 
International Standard and International Standard


Ces DeLat: Global Grid Forum (GGF) - modeled after IETF with minor 
differences
        four types of documents
        but no document has reached our second stage
        have to show interoperability and running code to progress
        GGF is 80 percent researchers, so specs lag review pretty badly
        working groups should shut down as soon as possible 
(sometimes too soon)


Steve Woodlock: Open Group (see slides)
        10-week timed process after draft iteration
        company review process and change requests followed by voting         
board approval


Tom Taylor: ITU-T
        The ITU-T has Study Groups, made up of working parties and 
questions
        Recommendations are the outputs of the ITU-T process
        all decisions on recommendations are made at meetings
        The document editor just edits
        the process is driven by written contributions
        last call issues when recommendation when "finished", 
last-call started by working parties
        if consensus on last-call, study group just approves the 
document
        if no consensus, try again at a higher level with 
governments as the voters
        maintenance - recommendations issued as new versions, 
identified by date assigned to a question for maintenance


Stephen Hayes: 3GPP
        the 3GPP process is either 1-stage or n-stage, conceptually
        one type of document - Technical Specification
        increase in functionality over time - version number 
increments
        three-level version numbers
        major updates every 18 months
        strong rules about backwards compatibility, but things do change 
over time
 
Alex Conta: noted that some of these organizations are based on 
corporate membership this is a radical difference with the IETF
        IEEE has similar process
        starts with contributions, goes through drafts, then to ballots in WG 
and final committee approval
        membership requirements are very strict (based on 
individuals) document changes include patches of earlier specs


4/ proposals for alternate standards track processes


4a/ draft-dawkins-pstmt-twostage-01.txt - Dave Crocker (see 
presentation)


Issues include barriers, unused stages, unused process, false 
advertising, uncoordinated use of drafts, and cruft in the archives.
Dave proposes a track that includes 3 stages: Working Group Snapshot, 
Proposed Standard, Internet Standard
        PS requires running code, but just one implementation
                PS is technical vetting, Internet Standard is a 
popularity
contest
        No requirement for multiple implementations - one widely 
deployed implementation can be a standard
        PS goes away after 36 months
        Working Group Snapshot is same as Scott's, working group 
synchronization, but IESG comments, doesn't approve totally under 
control of working group


Pete Resnick: for IS, does "community adoption" mean one 
implementation?
  Dave: yes, although it sounds crazy...


Scott Bradner: "it goes away?"
  Dave: goes to Historic (RFCs don't go away)


John Leslie: can you get from ID to PS in less than 6 months?
  Dave: it's an ID, it can be handled like an ID


4b/ draft-bradner-ietf-stds-trk-00.txt - Scott Bradner (see 
presentation)


IESG has changed the bar for Proposed Standard - doesn't let documents with 
nits go to Proposed Standards
Vendors don't wait for PS, and implement from random Internet drafts 
instead.
There is no significant difference between draft standard and full 
standard.
The effect is that the whole process has been shifted one step to the 
left.
Scott proposed a 3 stage process starting with a "stable snap shot" (SSS)
- level of IESG review is very low - a SSS is specifically a immature 
prestandard specification with no specific publication 
requirements.
The other two stages are Proposed Standard, largely unchanged from the way 
the IESG currently implements it, and Internet Standard, which is a 
combination of the current Draft and Internet Standard.


Keith Moore: noted that the "no known technical omissions" 
requirement in 2026 means a lot more than it used to (security) in 
addition now there's an IESG provided nits list, etc.


Chris Bemondi: noted that I-D nits focuses on normative references, so 
lengthens publications queue, too


Bob Hinden: noted that we now have two stages of IDs, individual and 
working group draft


Christian Huitema: we have seen a creeping insertion of a new step in the 
process working group draft is now the old PS, modulo some things


Aaron Falk: are SSSs ephemeral?
  Scott Bradner: depends on the proposal, but in my proposal they are not 
ephemeral


Craig White: what's the "big I Internet"?
  Scott Bradner: A non-isolated IP network


4c/draft-iesg-hardie-outline-00.txt - Leslie Daigle (see 
presentation)
   
draft-iesg-hardie-category-descriptions-00.txt


Leslie Daigle , speaking for Ted Hardie (who was tied up in another 
session), described Ted's proposed category descriptions.  His proposal 
uses a cooking analogy (ingredients, baking, baked, eaten, boiled)
        "Boiled" is "cooked by another process" - Experimental, etc.
        "Baking" could be archival
        one could also loosen lockstep reference requirements
        difference in archival series new archival series, "Candidate 
Specification"
new designation, "Full Standard" for documents that prove 
interoperability Aside - might want to talk about sharing RFC series 
(includes non-IETF documents)


Bob Hinden: observed that Leslie was doing a lot of cooking, but we're 
starving to death :-}. He noted that one of the problems we have is WG 
thinks it's finished, IESG doesn't agree
  Leslie: Ted also has document on document approval


Kurt Zeilenga: asked are individual submissions processed different?
  Leslie: Ted also proposes a variety of working group types


Aaron Falk: what about "spoiled" category?
  Leslie: working group work items are an archival series



4d/ draft-loughney-what-standards-00.txt - John Loughney (see 
presentation) 


John observed that the IETF has produced lots of RFCs, but not a lot of 
standards
Mobile Phone TCP team that just used STD 7/RFC 793, and was not sure what 
RFCs to use.
There are bugs in published specs - is there an errata process? new 
working group? what do you implement?
No way to figure out what the best version of a protocol is...
We just produce more specifications.
He offered some suggestions for things that might help: Better web 
interface? Maintenance teams? Early STD teams? Enhanced STD numbers?


4e/ Maintaining Standards - Brian Carpenter


Brian mentioned that he was trying to deprecate a feature in a draft 
standard but that the process was not a clean as it might be - How should 
one document a feature that is deprecated?  As a SHOULD NOT or a MUST Not? 
IETF technologies do not have version numbers which can make it easier to 
know what text people are referring to.  Fortran knew how to deprecate 
features - a feature was listed in a version of Fortran as deprecated and 
would not appear in the next version.  Vendors would say what version they 
supported so the user could know what features were present.  TCP is the 
extreme example of the confusion with IETF standards. Brian would love to 
have a one-step standards process with version numbers and a clear 
mechanism for identifying interoperability


Scott Bradner: remember the IPR licensing "feature" of Draft Standard when we 
evaluate proposals


Spencer: What about conditional IPR statements based on standards 
status/standards track?
  Brian: The new IPR procedure is better, but we still need to nail this 
down.


Dave Perkins: Re: versioning - what about references to specific 
versions?
  Brian: ASN had that problem - we need to understand this aspect of 
normative references, we need to be able to do both kinds of normative 
references


5/ what would define success in a revised IETF Standards track - Scott 
Bradner (see presentation)


Scott listed some things measures that might indicate that a revised 
standards process was a "success":
        more advancement along the standards track
        fewer products based on random I-Ds?
        better understanding within IETF/WG of document status?
        trade press understanding of document status?


Aaron Falk: documents are in state that corresponds to their status now.
Is 43 implementations of a PS is a failure? If it happens all the time, 
yes.
But the effort required for a change is proportional to size of change and 
this could be a big change.


Keith Moore: names of standard types need to coorrespond to reality. 
Particularly standards must be of high quality.  We need to be clear on the 
kind of endorsement we are providing.  And we need to be clear on 
currency - "relevancy" of the standard.


Alan DeCock: define success in terms of failure - should there be a 
failed documents archive?


Harald Alverstrand: the success of the IETF is in producing standards that 
make the Intenet work. We don't do this for fun. Guessing what to 
implement, and getting it wrong, is a failure. Our metrics should 
correspond to this.


Mark Handley: harald said that success is that the Internet works better but 
it works fairly well on Internet Drafts and Proposed Standards - they must 
have clarity and relevance.


Bill Strawn: Te current standards track isn't too difficult to 
understand and we should not care about the press confusion. We have a lot of 
problems - how many are the result of slow publication process?


Elliot Lear: success is when you take a TCPdump, identify the 
protocols that are running, and show they are documented in RFCs


John Loughney - There is no objective measure of success. There will be 
problems - success is in the eye of the beholder. There is some problem 
with the current process - we may not agree on what it is, but 
relevance of the work we do is what matters.


Dave Crocker: This environment is too solution-rich!  There are too many 
possibilities to tweak - what needs change badly? Media silliness 
doesn't do any damage, and we can't fix it anyway.  Cruft in archives is 
more of a problem. We should minimize the number of changes to minimize 
risk.


Christian Huitema: - He really likes Scott's proposal, which he finds the 
most realistic, although the other proposals are not very different. 
Christian observes that there is a danger in defining our work as 
"preventing harm to the Internet". This is a negative statement that leads to 
censorship. The IETF pendulum has swung a long way from permitting free 
experimentation to only allowing publication after extremely careful 
reviews; we need to swing back the other way. Currently, the "word on the 
street" is that the IETF spends a lot of time doing nothing. If we don't 
make it possible for people to agree without two years of 
censorship, we will fail


Bob Hinden - The IETF has stopped building the Internet, but new 
protocols are still be added independent of what the IETF does.  The 
control being asserted by the IESG has not worked (except to make the IETF 
less effective).


Harald:  Making the Internet work better means experiments, things we can 
do, fun doesn't mean blocking stuff, it means doing stuff.


6/ open discussion


Scott asked if there is consensus in the room that we need a change - the 
hum was clearly "yes"


Brian Zorn: "success of the IETF" in my own terms - "less trouble"  Two of 
the protocols I'm working on are I-Ds, there are multiple 
implementations and shipping products, nobody implemented the RFC. If it's 
not on time, don't even bother. We can't change protocols between I-D 
versions because it's already shipping. What about Individual 
submissions? Brian feels that we need version numbers, with 
interoperability statements. When a protocol takes years to come out - the 
people who want to use it have surrendered and left the building. We need to 
balance too fast and too slow, i.e. implementing from IDs vs. waiting for 
PS.

Melinda Shore: Melinda asked about retrieval - standardization process 
isn't doing archiving, everything should be archived and 
retrievable. She thinks we need protocol specification version numbers.  She 
feels that choosing working group documents is full of whimsy today, and 
should stay that way. What do you need to read to understand what a 
working group is working on - the charter is insufficient. Building a 
standards track around what particular documents are working group 
documents is the wrong thing to do.


Keith Moore: Is something wrong with our labeling? probably, but 
something is wrong with our process - we are iterating toward 
perfection - too many working groups are in FIN wait. Working groups need to 
shut down so they don't start doing strange things.  The document 
process needs to reflect this. Don't leave out Individual 
Submissions - they have to be accommodated. We need some kind of 
waterfall model for all this stuff - don't make IESG evaluate 
clean-sheet proposals with no experience. Maybe we should have sketches 
before snapshots?  WGs shoot down incomplete specs and don't read 
complete ones, so we read powerpoint!


Louis: The customers of these documents don't care about labels. IDs used to 
be "ideas." The market picks what they want to deploy.  Deploying 
Internet Drafts is fine, except for non-archiving. The IETF doesn't make 
Internet work, but it's where good work gets done. Make some tweaks and 
don't change much - adding process just won't happen.


Elwyn: Re: backwards compatibility - would my 1995 Sun/3 work today? 
Should we require backwards compatibility for standards at some level? 
Pre-standard Ethernet and modem vendors offered guaranteed upgrades to 
standard versions.


Scott Bradner: How do we make changes to standards-track protocols when new 
technology comes along if we focus too much on backward 
compatibility?


Elwyn: We need to be careful about what we standardize.


Alex: Two elements need to be addressed - number of stages and moving 
between stages - everybody has stages, no matter where you go. We need 
interoperable implementations. Moving between stages doesn't work well at 
IETF. Labeling could be radically improved - no indication of 
protocol, standard level, version number.


Jonathan Sadler: do we need to change the stages of normative 
references?


Kurt: we should ensure (as RFC 2026 does) the any revised IETF process 
allow individuals to produce any and all formal IETF outputs (e.g., all 
classes/categories of RFCs).


Brian Zorn: protocols that were individual submissions, but shouldn't have 
been - they were not WG documents when even though a WG said yes the WG 
charter said no. But companies are shipping the technology anyway.


Aaron: we should have multiple stages with interoperable 
implementations - we don't do conformance testing.


Alex: We should have at most two stages, we need the IPR hook, we need 
synchronization between IETF formality and Internet reality, there's a huge 
disconnect here. We need a lower bar with cross-functional review.


Mark Handley: How can we make the Internet work better? we should 
discourage implementation and encourage experimentation, while 
discouraging deployment. Mark notes that one can't tell stages of a draft
- no idea what attributes an I-D has under current process (e.g., 
reviewed   by congestion police).


Alex: Alex says that since documents are only advanced in meetings maybe we 
need more than three meetings per year?
  Scott Bradner:  Work is also done on mailing lists, no need for a 
meeting to advance a document


SUMMARY - 
        Scott asked for a quick show of hands to gage the feeling in the 
room on a number of topics:  strong support were shown for the ideas that:

                change is needed
                some type of working group snapshot is needed
                multiple implementations needed to show document clarity
                some kind of "IPR hook" is needed
                that a one stage process was not enough


Scott asked Harald, as AD, to take over
Harald: Do we have a rough idea of what we want to do? Yes, we need more 
discussion for WG.


Melinda: We really do know what the question is, just not the answer. We 
need more openness with other changes under consideration.


Bob Hinden: go for a WG now, there is a problem to solve


Craig White: Should we go back to interoperability testing? What is 
running code? We have no clarity about this


Alex: know what the problem is and have suggestions on how to fix it


Mark Handley: We have strong consensus change is needed, need to address it 
somewhere


Harald asked: can we write a WG charter now? 
        the feeling in the room was "yes"


Harald asked for volunteers for WG charter team and asked those 
volunteers to send a note to sob@harvard.edu.  He also said that we'll 
start a mailing