Rework RESERVED_RAW_IDENTIFIER and RESERVED_RAW_LIFETIME

This reworks the reserved raw identifier and lifetimes to hopefully more
clearly express what they mean.

The "except" clauses in the raw identifier were intended to mean a set
subtraction, not an explicit "and it is an error if it is specified".
Using set subtraction isn't correct because that would mean `r#crate`
would be interpreted as 3 tokens (since RAW_IDENTIFIER did not match it,
but IDENTIFIER_OR_KEYWORD PUNCTUATION IDENTIFIER_OR_KEYWORD would).

I also reordered Token, since the intent is that the first production
in an alternation that matches wins. The idea here is to make the
reserved tokens high priority, so that they clearly match first and
cause an error. (I did not exhaustively analyze the rest of the rules
to see if they follow that behavior, that is for another day.)

It could be said it would be nice to document the rationale for the
restrictions (#2042), but
that is a bigger ask.

Author: ehuss

src/identifiers.md

@@ -9,13 +9,13 @@ XID_Start -> <`XID_Start` defined by Unicode>
 
 XID_Continue -> <`XID_Continue` defined by Unicode>
 
-RAW_IDENTIFIER -> `r#` IDENTIFIER_OR_KEYWORD _except `crate`, `self`, `super`, `Self`, `_`_
+RAW_IDENTIFIER -> `r#` IDENTIFIER_OR_KEYWORD
 
 NON_KEYWORD_IDENTIFIER -> IDENTIFIER_OR_KEYWORD _except a [strict][lex.keywords.strict] or [reserved][lex.keywords.reserved] keyword_
 
 IDENTIFIER -> NON_KEYWORD_IDENTIFIER | RAW_IDENTIFIER
 
-RESERVED_RAW_IDENTIFIER -> `r#_`
+RESERVED_RAW_IDENTIFIER -> `r#` (`_` | `crate` | `self` | `Self` | `super`)
 ```
 
 <!-- When updating the version, update the UAX links, too. -->
@@ -72,7 +72,7 @@ Unlike a normal identifier, a raw identifier may be any strict or reserved
 keyword except the ones listed above for `RAW_IDENTIFIER`.
 
 r[ident.raw.reserved]
-It is an error to use the [RESERVED_RAW_IDENTIFIER] token `r#_` in order to avoid confusion with the [WildcardPattern].
+It is an error to use the [RESERVED_RAW_IDENTIFIER] token.
 
 [`extern crate`]: items/extern-crates.md
 [`no_mangle`]: abi.md#the-no_mangle-attribute

src/tokens.md

@@ -4,7 +4,7 @@ r[lex.token]
 r[lex.token.syntax]
 ```grammar,lexer
 Token ->
-      IDENTIFIER_OR_KEYWORD
+      RESERVED_TOKEN
     | RAW_IDENTIFIER
     | CHAR_LITERAL
     | STRING_LITERAL
@@ -18,7 +18,7 @@ Token ->
     | FLOAT_LITERAL
     | LIFETIME_TOKEN
     | PUNCTUATION
-    | RESERVED_TOKEN
+    | IDENTIFIER_OR_KEYWORD
 ```
 
 r[lex.token.intro]
@@ -769,9 +769,9 @@ LIFETIME_OR_LABEL ->
     | RAW_LIFETIME
 
 RAW_LIFETIME ->
-    `'r#` IDENTIFIER_OR_KEYWORD _except `crate`, `self`, `super`, `Self`, `_` and not immediately followed by `'`_
+    `'r#` IDENTIFIER_OR_KEYWORD _not immediately followed by `'`_
 
-RESERVED_RAW_LIFETIME -> `'r#_` _not immediately followed by `'`_
+RESERVED_RAW_LIFETIME -> `'r#` (`_` | `crate` | `self` | `Self` | `super`) _not immediately followed by `'`_
 ```
 
 r[lex.token.life.intro]
@@ -786,7 +786,7 @@ r[lex.token.life.raw.allowed]
 Unlike a normal lifetime, a raw lifetime may be any strict or reserved keyword except the ones listed above for `RAW_LIFETIME`.
 
 r[lex.token.life.raw.reserved]
-It is an error to use the RESERVED_RAW_LIFETIME token `'r#_` in order to avoid confusion with the [placeholder lifetime].
+It is an error to use the [RESERVED_RAW_LIFETIME] token.
 
 r[lex.token.life.raw.edition2021]
 > [!EDITION-2021]
@@ -922,7 +922,7 @@ r[lex.token.reserved]
 ## Reserved tokens
 
 r[lex.token.reserved.intro]
-Several token forms are reserved for future use. It is an error for the source input to match one of these forms.
+Several token forms are reserved for future use or to avoid confusion. It is an error for the source input to match one of these forms.
 
 r[lex.token.reserved.syntax]
 ```grammar,lexer
@@ -1058,7 +1058,6 @@ r[lex.token.reserved-guards.edition2024]
 [numeric types]: types/numeric.md
 [paths]: paths.md
 [patterns]: patterns.md
-[placeholder lifetime]: lifetime-elision.md
 [question]: expressions/operator-expr.md#the-try-propagation-expression
 [range]: expressions/range-expr.md
 [rangepat]: patterns.md#range-patterns