Review panel's role

[Pekka Savola <pekkas at netcore.fi>]

On Thu, 4 Aug 2005, John C Klensin wrote:
> (2) Several comments, during and after the discussion and most
> precisely framed by Spencer Dawkins, that I may have made an
> incorrect assumption about transition.   The text more or less
> assumes that the review panel membership would be new
> and the IESG membership would  be left in place.   Perhaps the
> current IESG membership are most skilled in document review
> rather than the sort of WG leadership and steering functions
> that I had in mind -- the functions I think we had in the
> early 1990s.  If that were the case, then we should resolve the
> detail of the IESG being larger than the review panel, shift
> the current IESG members onto the review panel, and repopulate
> the steering/coordination/management entity (probably calling it something 
> other than "IESG").

Personally, I don't think changing the proposal in this aspect is 
worth the effort.  Changing people around seems more trouble than 
electing new ones.  The existing IESG people, if they felt the review 
panel is more suited for them, could very well apply to the review 
panel, and if selected, the nomcom would pick the IESG replacement. 
In a year or two, everything would be settled in any case.  The people 
will find the roles they're interested in.

(But this wasn't my maint point in replying..)

> (3) A comment from an IESG member that the notion would probably add work, 
> since the document provides for documents rejected  by the review panel to 
> cycle back through the IESG. To me, this is one of the "sample detail" issues 
> identified above.  If the IESG wants to be explicitly in that loop, and the 
> community wants that, then "back to the IESG as the submitting body" is the 
> right model.  I had anticipated their involvement at that point as 
> lightweight, with the AD reviewing the comments and passing them on to the 
> WG, but not assuming today's role of negotiator with the WG (or between the 
> WG and the review panel).  I think that negotiator role, after a document 
> goes out for IETF Last Call, is the source of several of our problems and 
> needs to be eliminated.
>
> For those who are reading this without having read the document, please note 
> that rejection of a document by the review panel is intended to be A Big 
> Deal.  The document suggests a process that would focus on getting 
> good-quality, pre-reviewed, documents to the review panel --with shared 
> responsibility between the WG and the IESG's steering/managing function for 
> being sure that happens-- with the review panel only organizing a final 
> check.

I'm troubled with the assumption that the review panel rejection is A 
Big Deal.  This has unstated assumptions on what kind of people 
you'd expect to be on the review panel and/or what kind of review is 
expected.

As an occasional reviewer myself, a few observations:
 * All the indepth reviews I have done have resulted in issues,
 * If I haven't found a problem or a request for clarification, I
   haven't paid sufficient attention to the review, and
 * IMHO there is nothing more frustrating than finding problems (not
   maybe critical ones) which don't get fixed for one reason or
   another (e.g., because it isn't worth the time to respin the
   document, giving that comment would be a "big deal", ...).

Scientific reviews certainly have "accept", "reject", or "accept with 
modifications".

Maybe forcing "yes, business as usual" or "no, A Big Deal" is 
intentional -- to encouraging the reviewers to perform less in-depth 
reviews (with more responsibility on the chairs and the IESG) so that 
they'd only complain of the most horrid problems, to lessen the load, 
to encourage earlier participation, make sure the majority of the docs 
go through without issues, etc.

As how to address this, a few observations:
 * the review panel's review function has to be roughly equivalent to
   what review the IESG is doing today, so that the IESG can
   "give up" most of its review with good confidence.
 * to achieve this, I think we need "accept", "reject",
   "tentative accept" resolutions where in all cases the panel could
   include feedback on things that were noted.
 * whether or not the review panel (or the reviewers) check the
   revised documents after "tentative accept", whether the checking
   responsibility is with the AD(s), or something else is
   an interesting detail point which probably isn't relevant at this
   point.

Hope this helps in getting a better understanding on what the role of 
the panel could be.

--
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

_______________________________________________
Ietf mailing list
Ietf at ietf.org
https://www1.ietf.org/mailman/listinfo/ietf