Tracking issue for migrating from Glacier to compiletest ICE/crash tracking

[fmease]

Tracks the necessary steps to "fully"¹ transition from rust-lang/glacier to compiletest ICE/crash tracking inside tests/crashes/.
See also this Zulip topic.

Steps

• Implement rustc ICE/crash tracking: compiletest ice tracking #122997
• Implement rustdoc ICE/crash tracking
• Implement rustfmt ICE/crash tracking (questionable)
• Implement ICE/crash tracking via run-make/rmake (to supersede Glacier's shell scripts)
• Archive rust-lang/glacier: 🧊 Put glacier on ice 🧊 team#1459
• Sunset triagebot support for Glacier
• Update the rustc-dev-guide
  • Remove mentions of Glacier
  • Add small section about tests/crashes/: Overview, rough procedure (adding //@ known-bug, S-bug-has-test Status: This bug is tracked inside the repo by a `known-bug` test. )
  • Mention tests/crashes/ in places where we reference E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added.
• Update the Rust Forge
  • Remove mentions of Glacier (namely the triagebot section)
  • Maybe mention the new system in the issue triaging chapter
• If technically feasible, add a CI check that a PR fixing crash tests mentions all fixed issues via GH auto-close keywords like Fixes to prevent the accumulation of "ghost issues".

Unresolved Questions

• Should we implement rustdoc support via subdirectories/tests suites (e.g., tests/crashes/rustdoc/) or via a new compiletest directive (e.g., //@ bin: rustdoc)
  • If we do the former, should we move rustc tests from tests/crashes/ to tests/crashes/rustc/?
• Should we define precise procedures for adding new tests to tests/crashes?

Footnotes

1. Glacier|_{rust-lang/rust}, i.e., restriction of Glacier to rust-lang/rust (excluding git submodules/subtrees), i.e., Glacier without support for rustfmt, Cargo, Clippy, Miri ICE/crash tracking ↩

[fmease]

cc @matthiaskrgr

[matthiaskrgr]

I'm not sure if it makes a lot of sense to track crashes inside the rustc repo for tools (rustfmt, clippy, miri etc) that are actually living inside their own repos because in most cases I guess you would "accidentally" fix something during a submodule/tree sync?

[fmease]

Good point. I guess that means T-rustfmt/T-cargo/... for example will need to introduce their own mechanism assuming they're interested in doing that and that "we" won't try to find a Glacier replacement for them now that it's officially archived.