Subject: Last Call: <draft-koster-rep-06.txt> (Robots Exclusion Protocol) to Informational RFC

From: The IESG <[email-masked]>
Date: Mon, 28 Feb 2022 09:39:48 -0800

The IESG has received a request from an individual submitter to consider the
following document: - 'Robots Exclusion Protocol'
  <draft-koster-rep-06.txt> as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2022-04-07. Exceptionally, comments may
be sent to [email-masked] instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract

   This document specifies and extends the "Robots Exclusion Protocol"
   method originally defined by Martijn Koster in 1996 for service
   owners to control how content served by their services may be
   accessed, if at all, by automatic clients known as crawlers.

The file can be obtained via
https://datatracker.ietf.org/doc/draft-koster-rep/

No IPR declarations have been submitted directly on this I-D.