Re: draft-housley-two-maturity-levels

SM <sm@resistor.net> Mon, 05 July 2010 21:03 UTC

Return-Path: <sm@resistor.net>
X-Original-To: ietf@core3.amsl.com
Delivered-To: ietf@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C329328C0D8 for <ietf@core3.amsl.com>; Mon, 5 Jul 2010 14:03:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.392
X-Spam-Level:
X-Spam-Status: No, score=-1.392 tagged_above=-999 required=5 tests=[AWL=-1.207, BAYES_40=-0.185]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sNFKoclfWYWG for <ietf@core3.amsl.com>; Mon, 5 Jul 2010 14:03:20 -0700 (PDT)
Received: from ns1.qubic.net (ns1.qubic.net [208.69.177.116]) by core3.amsl.com (Postfix) with ESMTP id 2B3603A67AF for <ietf@ietf.org>; Mon, 5 Jul 2010 14:03:20 -0700 (PDT)
Received: from SUBMAN.resistor.net ([10.0.0.1]) (authenticated bits=0) by ns1.qubic.net (8.14.5.Alpha0/8.14.5.Alpha0) with ESMTP id o65L2e9h013878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Jul 2010 14:02:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=opendkim.org; s=mail2010; t=1278363774; x=1278450174; bh=ok+H5qDnAUIdsRTuFhjC/8QaKMSr9LvAJ4jJcTw9oJ4=; h=Message-Id:Date:To:From:Subject:Cc:In-Reply-To:References: Mime-Version:Content-Type:Cc; b=W8WDyr2+Woc/0NoauurqdaaCX1XvQ/mCt1E034eqJz51GcXBESTEO0CRNPk831g4R MU/FTGS1wqPmNdroCNhjxFBPloBCQyCMCnjpjsi3bVf9zHgtvsSZzfI4YHTGzqI5vv paF+ClsC7aKNybDoN5wdQ2Hn1nLPv3xsLrwxEiho=
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=resistor.net; s=mail; t=1278363774; x=1278450174; bh=ok+H5qDnAUIdsRTuFhjC/8QaKMSr9LvAJ4jJcTw9oJ4=; h=Message-Id:Date:To:From:Subject:Cc:In-Reply-To:References: Mime-Version:Content-Type:Cc; b=c6EfTfeM5fT0ZLNQJid9VsPnxrZQSQYnvLvDAEXfczSFk2yUc0umZUy9Y1ME0PVzB +pDduMP7ZMzMPOraoN+kRY+cvsqGxK+qI2lgiM535pzZQKkgl7NceckxeYIxhJnEwb yiGcgavXiWUiVwn3dbN+IJwo7kMP7VJojIiZ1wtY=
DomainKey-Signature: a=rsa-sha1; s=mail; d=resistor.net; c=simple; q=dns; b=X72of3PaSxJ46bBEeQwal/pzZJVzRBBeTz/NflZ1krfTEPsnhfY08ctyQZzdHrRfI ssktTiO2HwVda5VfKnr7H3/OcFxLYu1B4kpoJ6ScGGmlY6MnShBBYicTsLvpn6o50Cb TJcDZrZbV5M8zMkjspjYpv3cTybX8ZMMCpqifqA=
Message-Id: <6.2.5.6.2.20100705121135.0accf848@resistor.net>
X-Mailer: QUALCOMM Windows Eudora Version 6.2.5.6
Date: Mon, 05 Jul 2010 13:49:05 -0700
To: John C Klensin <john-ietf@jck.com>
From: SM <sm@resistor.net>
Subject: Re: draft-housley-two-maturity-levels
In-Reply-To: <599DEF8BAF6E7D88746D18C4@[192.168.1.128]>
References: <599DEF8BAF6E7D88746D18C4@[192.168.1.128]>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format="flowed"
Cc: ietf@ietf.org
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 05 Jul 2010 21:03:22 -0000

Hi John,
At 17:38 04-07-10, John C Klensin wrote:
>(1) Analysis of the problem as stated in
>draft-housley-two-maturity-levels-01
>
>The draft indicates (Section 1) that:
>
>         "The proposed changes are designed to simplify the process
>           and reduce impediments to standards progression"
>
>and then
>
>         "Since many documents are published as Proposed Standard
>            and never advances (sic) to a higher maturity level, the
>           initial publication receives much more scrutiny that is
>           call (sic) for by RFC 2026 [1]."

There is also:

   "In practice the annual review of Proposed Standard documents after
    two years has not taken place."

I understand that the ADs have a lot of work.  Why it took 14 years 
for someone to say that the annual review is not being done is a mystery.

>However, if the issue is that documents rarely progress beyond
>Proposed, then there is no evidence that tampering with the
>relationship between the current Draft and Full Standard levels
>will bring about any change in that relationship, i.e.,
>increase the number of documents that move past Proposed.  I
>note, as others have noted, that the current de facto
>requirements for Proposed are far in excess of what 2026
>requires.  If the problem is (as I believe it is at least in
>part) that initial publication receives far too much scrutiny
>and takes far too much time and energy, then formally no
>procedural changes at all are needed: the IESG simply needs to
>stop doing that and instead and apply only the level of
>scrutiny that RFC 2026 requires, and no more.  Such a change
>would result in more rapid publication of protocol documents as
>Proposed Standards (perhaps reducing the fraction of the
>Internet that actually runs on I-Ds) so the community can
>evaluate whether reducing the community exhaustion from getting
>things to Proposed results in more documents moving to the next
>level (whether that is one next level or one of two.

Agreed.

>It appears to me that there are portions of the IETF community
>who are willing to advance documents to Draft.  Most of that

Yes, until they are asked whether it is worth the bother.

Here are some quotes from the IETF 76 plenary:

  "Loa Andersson mentioned a working group that has been around for 13
   years with one RFC that is at Draft Standard. We don't have a process
   problem, we have a mindset problem. There is too much work required
   to get a document to Full Standard."

  'Ralph Droms has chaired a working group for 20 years with two documents
   at Draft Standard and the rest at Proposed. When considering progression
   on the maturity ladder, Ralph was once asked by an AD "is that really
   where you want to spend your time?"'

  "Larry Masinter asked do the documents accurately reflect what is
   implemented and deployed? Lars Eggert responded why doesn't the
   IESG get this question? What do you want the IESG to do? Larry
   answered that he wanted a clearer process for defining a document for
   Full Standard. Pasi Eronen reminded folks that the reason we are
   getting these questions is because we badly screwed up the handling of
   first draft from the YAM working group. We have a better idea going
   forward."

>Those who have been anxious to advance documents within the
>existing model have often met significant resistance from IESG
>members.  It is somewhat disingeneous for the IESG (or even
>some of its members) to resist efforts to move documents to
>Draft or Full Standard, to place requirements on approval of
>Proposed Standards that go well beyond those of 2026, and then
>to complain about how few documents advance to Draft or Full
>Standard and use those complaints as the justification for
>abolishing the Full Standard level.

Based on publicly available information, I would say that the answers 
lies in the IESG evaluation of 
draft-ietf-yam-5321bis-smtp-pre-evaluation-04.  The draft is a path 
that precludes any appeals or recalls.  It kills two birds with one stone.

>On the other hand, the analysis in the document ignores the
>observation that, independent of the interoperability
>demonstration, moving a specification up in maturity levels
>often results in significant improvement in document quality
>and clarity.  Again, it might represent an even larger
>improvement if the IESG more closely adhered to what many of us
>believe is the intent described in RFC 2026 and earlier
>documents -- to get a specification published for examination
>and implementation purposes as soon as all known technical
>defects have been identified or eliminated but without
>investing the resources to polish the text editorially.  See
>(3) below for more discussion on that subject.

The IESG does a lot of work polishing drafts because of the quality 
of some of the drafts that gets to IESG evaluation.  The work that 
the IETF Community should be doing is being pushed up to the 
IESG.  The end-result is that the intent in RFC 2026 has fallen into disuse.

>Conclusion: Whatever this proposal may or may not accomplish,
>there is no evidence that it will solve the problems identified
>in its Introduction.

You mean like "since we are not using this mechanism, let's get rid 
of it instead of thinking why the mechanism is there".

>(2) Promoting more use of Draft Standard (under any name) by
>eliminating Full Standard and renaming.
>
>YMMD, but I believe that the fraction of the community who now
>believe that having something at Proposed (1st step) only is a
>bar to implementation is fairly small.  If it weren't, we'd be
>in bad trouble as a consequences of the very active use of a
>number of protocols that have never advanced beyond Proposed.
>It is possible --although I have some doubts-- that rebranding
>would help here.  But, to the extent to which the need is to
>rebrand, that does not, in itself, justify changing the
>maturity level model.

I doubt that re-branding can change a mindset.

>If one simply wanted to rebrand to reflect present-day reality,
>we would change Proposed/ Draft/ Standard to Standard/ Standard
>with Gold Star/ Standard with Gold Star and Network Cable
>Wreath.  Those titles are somewhat in jest, but the point is
>that one wants to make the first level "Standard" and then
>start talking about endorsements of higher specification
>quality or assurance (see #6 below).

Yes.

>Our real terminology/ branding problem is that "Proposed
>Standard" has periodically been interpreted -- sometimes by
>people or organizations with other agendas -- as "substandard
>and really not suitable for deployment".  In fairness to them,
>that was exactly our intention in the early 1990s.  That
>intention was very much part of the "upgrade or die" language
>in 2026, but has never been used.  Leaving "Proposed Standard"
>as part of our vocabulary doesn't solve that problem.

One alternative is to remove "Proposed Standard" from the vocabulary.

>more information and discussion.  I also note that BCP numbers
>are sometimes used to cluster groups of documents just as STD
>numbers are. In both cases, there may be issues with idea and

Yes, and that seems to have been forgotten.

>(3) Conflating protocol quality with document quality
>
>Advancing documents according to the current model actually
>provides two separate (and separable) functions.  One is
>certification of some minimal proof of interoperability (Draft)
>and utility (Full).  The other is the specifications almost
>always undergo significant improvements in the quality of their
>documentation and description as people are required to
>re-examine and re-edit them in the process of advancing between
>grades.  I suggest that the IESG is not taking enough advantage

If we expect clear, concise, and easily understood documentation, we 
should re-examine the documents and re-edit them if necessary while 
not destabilizing the protocol.

>Either way, we should have a high document-quality expectation
>at the second (and, if it exists, third) maturity levels.

Yes.

>(4) Downward references
>
>I don't see the evidence for "major cause of stagnation in the
>advancement of documents" unless one relaxes the rule to permit
>normative references to I-Ds (which I do not favor).  If the
>IESG believes that a document is being stuck unreasonably (or
>is likely to get stuck), it has the ability to invoke the RFC
>4897, which merely requires asking the community (in a Last
>Call that can be combined with the Last Call on the document if
>desired but that can also be applied later) and noting the
>downref in the document.  That is not an onerous procedure, yet
>the IESG has never chosen to use it.  If we really want to

The problem is that the IETF has been publishing BCPs that are never 
implemented because they got forgotten along the line.

>treat standards at the second maturity level (or third, if it
>remains) as better and more completely-specified than those at
>lower levels then the downref restriction should remain, with
>the community able to make exceptions when the lower-level
>document(s) are considered sufficiently mature and
>well-specified.  If, by contrast, additional maturity levels
>are endorsements indicating that almost-orthogonal quality
>levels have reached (as in the "gold star" recommendation of #2
>above), then the downref issue may become irrelevant.
>
>Conclusion: This proposed change solves a problem that has not
>been demonstrated to exist, even to the extent of demonstrating
>that the elimination of current mechanisms for permitting
>downward references would save the IESG significant time.

It does not matter if there are Downward References for "Proposed 
Standard".  If the RFC referred to is mature enough, the Downward 
Reference should not be an issue.  It's not like the rules say that 
"downward references are not permitted".

>Conclusion: Whatever its other strengths and weaknesses, the
>proposal is patchwork that ignores a major component of the
>situation and should not be approved until and unless that
>issue is addressed.  Suggestions on the IETF list, including,
>if I recall, some from IESG members, that the proposal should
>be approved in Maastricht are obviously inconsistent with the
>nature of the proposal and its relationship to actual cause and
>effect analysis.

It is political expedient to resort to patchwork to avoid opening the 
larger debate.  As we take a narrow view of the issues, we lose the 
bigger picture.

>(7) The process questions surrounding proposal, posting, and
>processing of this document.
>
>Independent of the content of this document, the procedure used
>to develop it, discuss it in the community, and presumably to
>reach conclusions about it seems to be to be seriously
>questionable.  It is identified as derived from an IESG
>discusson.  It was written by the IETF Chair and General Area
>Director, who has previously taken the position that process
>change proposals were unwelcome unless they met narrow
>categories, categories that were not widely exposed to the
>community much less the result of community consensus.  It has
>been scheduled for plenary time at IETF 78 (by the IETF Chair
>and/or the IESG) and announced with timing that makes
>preparation of well-developed alternate proposals difficult.  I
>note that the IESG discussions that led to this proposal
>apparently do not even appear in published minutes.  Even if
>the IESG review is ultimately "sponsored" by some other AD, the
>fact that this proposal affects the way the IESG does business
>and is derived from IESG discussions gives it the appearance of
>an IESG proposal, scheduled for discussion by the IESG (or an
>IESG member) under conditions set by the IESG or that member,
>and then subject to consensus review and approval by the same
>IESG.  This does not appear to me to be how we want to do
>things; certainly it calls the openness and transparency of the
>IETF into question unless we argue that IESG members are
>granted powers to make for the community and independent of any
>requirement for fairly-determined community consensus by virtue
>of appointment to their positions.

The author of the draft is the current IETF Chair.  I have some 
reservations about the IETF Chair driving such a proposal through the 
process.  Although the IETF Chair is also an IETF participant, it can 
be perceived as problematic when the person writes a non-technical 
proposal that has to be evaluated by the IESG.

This matter was discussed at the IESG retreat and the discussions 
have not be published.  I am not asking for them to be 
published.  But there should be a record in the IESG minutes if the 
"Internet Engineering Steering Group (IESG) discussed the possible 
ways of reducing impediments to standards progression".

I agree that draft gives the appearance of an IESG proposal.  Nobody 
would be foolish enough to write an alternative proposal.  If there 
were such a person, he or she would find it difficult to get an AD to 
sponsor the proposal.  According to AD Sponsorship guidelines, such a 
proposal would have to be sponsored by the General Area Director 
which happens to be the IETF Chair.

This is the second time in two months that we have seen such 
"procedures" being used.  Although there is nothing wrong with the 
motives, i.e. to solve a problem, the way it is being done is plainly wrong.

Regards,
-sm