I have done a quick review focusing only on HTTP related parts, and on general RFC style.

Style nits:

- please separate artwork for request / responses and tag them with "httpmessage"
- one use of "COULD" (that's not a BCP 14 keyword)
- please use "Section x of [REF]" instead of "[REF] section x"
- most artwork is too wide for plain text rendering 
- there's no reason to specify an abbreivated title when the title of the document is already short enough
- 6.2.1 has prose in artwork instead of plain text, thus the warning about RFC 8141 being unused


HTTP remarks:

- "The WHEP Endpoints MUST return an "405 Method Not Allowed" response for any HTTP GET, HEAD or PUT requests on the endpoint URL in order to reserve its usage for future versions of this protocol specification." - I think this is highly problematic. If you want to reserve specific uses for GET, defining media types for that purpose and doing content negotiation makes more sense

- there is a trend towards being incorrect or too specific with status codes; for instance, 409 sounds incorrect if the it's really about the endpoint not being ready; in which case a 5xx makes more sense. Furthermore, requiring 204 is too specific; there's really no reason why an empty 200 reponse shouldn't work as well. It's also not clear why "307" "MUST" be supported, and 308 is not mentioned at all.

- don't be too specific with additional requirements on requests/responses, in particular when they are already defined by HTTP (see, for instance, handling of ETags and conditional header fields)

In general, I recommend reading https://www.rfc-editor.org/rfc/rfc9205.html and to revise the document where it either is to specific, or tries to redefine that HTTP already specifies.