Miri does not support swapping out `core`

[gamozolabs]

I'm working on a custom core called corps which breaks some compatibility with Rust. While I realize I'm way off in the weeds and don't expect much support from things, I ran into an issue with miri where it resolves core::ffi::c_ulong as well as core::ascii::escape_default unconditionally, which causes miri to not work in my custom core.

"Fixing" the issue (temporarily for me) was quite a simple fix, and if miri was much further from working in my environment I wouldn't really bother making an issue.

The issues are in chunk_size in src/tools/miri/src/concurrency/cpu_affinity.rs which uses core::ffi::c_ulong, and the "sentinel" value checked in src/tools/miri/src/eval.rs of core::ascii::escape_default.

This is the diff I used to make miri work in my environment. Obviously, this is a hack workaround where I just remove the check if libcore is present in eval, and hard-code a c_ulong size. But I imagine that there would be a way to do these things through lang items which are accessible regardless of the path names.

diff --git a/src/tools/miri/src/concurrency/cpu_affinity.rs b/src/tools/miri/src/concurrency/cpu_affinity.rs
index 9583de5a483..0b6cd95a10c 100644
--- a/src/tools/miri/src/concurrency/cpu_affinity.rs
+++ b/src/tools/miri/src/concurrency/cpu_affinity.rs
@@ -32,10 +32,11 @@ pub fn new<'tcx>(cx: &impl LayoutOf<'tcx>, cpu_count: u32) -> Self {
         this
     }
 
-    pub fn chunk_size<'tcx>(cx: &impl LayoutOf<'tcx>) -> u64 {
+    pub fn chunk_size<'tcx>(_cx: &impl LayoutOf<'tcx>) -> u64 {
         // The actual representation of the CpuAffinityMask is [c_ulong; _].
-        let ulong = helpers::path_ty_layout(cx, &["core", "ffi", "c_ulong"]);
-        ulong.size.bytes()
+        //let ulong = helpers::path_ty_layout(cx, &["core", "ffi", "c_ulong"]);
+        //ulong.size.bytes()
+        8
     }
 
     fn set<'tcx>(&mut self, cx: &impl LayoutOf<'tcx>, cpu: usize) {
diff --git a/src/tools/miri/src/eval.rs b/src/tools/miri/src/eval.rs
index 4c531a8d1f5..ca179a8a486 100644
--- a/src/tools/miri/src/eval.rs
+++ b/src/tools/miri/src/eval.rs
@@ -9,7 +9,7 @@
 
 use rustc_abi::ExternAbi;
 use rustc_data_structures::fx::{FxHashMap, FxHashSet};
-use rustc_hir::def::Namespace;
+//use rustc_hir::def::Namespace;
 use rustc_hir::def_id::DefId;
 use rustc_middle::ty::layout::{HasTyCtxt, HasTypingEnv, LayoutCx};
 use rustc_middle::ty::{self, Ty, TyCtxt};
@@ -329,6 +329,7 @@ pub fn create_ecx<'tcx>(
         MiriMachine::new(config, layout_cx, genmc_ctx),
     );
 
+    /*
     // Make sure we have MIR. We check MIR for some stable monomorphic function in libcore.
     let sentinel =
         helpers::try_resolve_path(tcx, &["core", "ascii", "escape_default"], Namespace::ValueNS);
@@ -337,7 +338,7 @@ pub fn create_ecx<'tcx>(
             "the current sysroot was built without `-Zalways-encode-mir`, or libcore seems missing.\n\
             Note that directly invoking the `miri` binary is not supported; please use `cargo miri` instead."
         );
-    }
+    }*/

[lolbinarycat]

Miri has it's own repository and issue tracker, so i'm transferring this to there.

@rustbot transfer miri

[RalfJung]

Yeah Miri doesn't support no_core code or different core crates, sorry. We have to draw the line somewhere to keep maintenance manageable, as we are a tiny team. Introducing a lang item for everything we need is not practical. FYI on Linux targets we also assume that the libc crate is in the sysroot and will heavily use it for all sorts of types and constants.

[oli-obk]

Yea I think what we could do is stick all these paths into constant items and put all of them into one (or multiple for platform specific ones) modules that make it easier for you to just change them. But beyond that, we're tied to libcore.

[gamozolabs]

Yeah, definitely understood. As long as it's easy for me to patch/spoof out so I can continue using miri for my custom core I'll be pretty happy. I didn't know if this would help with Rust internals as I know they have a lot of tests with no_core that maybe would like using miri.

One thing that would make sense to me is if core could be derived from the crate that uses #![rustc_coherence_is_core] such that the paths at least would resolve to the crate that is behaving as core. I'm not sure on the special meaning of that marker though and how it's used in Rust.

But of course, I know this is definitely a fringe issue and don't expect anything! I might poke around and do a PR. I write a lot of atomic data structures and miri is the coolest tool ever <3

[bjorn3]

#![rustc_coherence_is_core] is not encoded in the crate metadata, so it is only accessible while compiling libcore itself. It allows adding implementations to incoherent_impls inside the crate metadata, but #[rustc_allow_incoherent_impl] on individual impls does the exact same, so I don't think that will work for miri detecting libcore.

[RalfJung]

We used to have some logic that was smarter about finding std based on some lang items or so, but there were problems so I removed it again... it might be worth digging up that history when considering new approaches.

Most of our assumptions about core/libc paths are inside shims, so you won't hit them unless you actually invoke those functions. Not sure how far you're planning to go with that alternative core, if it's more than pure Rust code then the patch you posted will not be enough, not by a long shot.

Your patch indicates the two places where we do such a lookup already during initialization (I think there's at least one more such place if you use a Windows target):

• The first thing you are hitting here, escape_default, is just a sanity check to give a nice error when people run Miri with the default sysroot (rather than the Miri sysroot). I would like us to keep that, since people will continue using the miri driver the wrong way and this catches one of the most common misuses. Removing that check as your patch does is definitely not something we'll land, but we could have a -Zmiri-skip-sysroot-sanity-check or so.

• The other one saves us from having to know how large "unsigned long" is on all targets. That is quite convenient indeed... but it is probably possible to delay initializing the CPU affinity masks (for which this matters) until one of the relevant shims is used. So this again boils down to whether you are just interested in pure Rust code, or whether you plan to eventually invoke C shims like open/close/sched_getaffinity from your code.

  Or we just say all libcore we support must have an appropriate ffi module, that doesn't seem unreasonable TBH. 🤷

[gamozolabs]

That lines up with what I gleaned from the patches I made. The reasons for those checks make sense.

I'm kind of okay with the patches I made locally right now. I haven't been looking too much as more reliable patches that could be merged upstream yet. Although it's something I want to look at just to make sure miri doesn't get harder to patch for my environment long term (my environment is pure-Rust, no third party crates, no libc, etc).

As far as miri support I really only care about being able to reach my programs main() (super easy, as I can just invoke it from the miri entry point). I just want to be able to make data structures like Map::new() and then experiment with operations to double check for safety issues. Not really looking to run "complete" programs in miri, rather just test my primitives (eg. no syscalls).

And as always, as a custom core I expect no support from Rust or miri, unless I reasonably think other things would benefit or the changes are so small (eg. I could see a miri-based test for a super minimal Rust-only #![no_core] test file).

corps Context

For some context on my core (if interested, really stretching the limits of Rust features):

It's something I've been trying to do since I started Rust maybe ~10 years ago, but only recently have I seriously put my head down on it and now I'm building projects with it. I've called it corps and it has some large departures from Rust in that it has no panics (and thus no unwinding), everything is const (literally everything, including syscalls, which will fail at const-time evaluation if hit, but not if reachable (eg. fail open)), and my "Result" type tracks source file/line/etc at each site that passes errors up so even in LTO-ed inlined code I still get a "fully readable" stack, since a lot of code I write simply takes too long to run in debug builds.

It's really me using Rust as a playground for some things I want to do in a programming language myself, but for now building off Rust is really nice!

It's complete enough that I've written a debugger, ELF parser, DWARF parser, etc in it. I made an injection stub which allows me to parse an ELF entirely at const-time such that the binary contains the "in-memory image" of the ELF baked in .rodata. Idk, it's really fun!

TL;DR:

• Fairly developed (file I/O, maps, vectors, format strings, atomics, etc)
• Threads coming soon, but will have a different model where you pick the number of threads as an attribute of main, and main will get invoked that many times with a thread ID. (TL;DR: no dynamically spawned/exited threads)
• x86_64 and arm64 support on Linux, with plans to build my own OS in it
• Absolutely no third party code or FFI. corps defines _start and directly executes syscall instructions
• Everything is const (including syscalls, const_eval_select causes an error if hit)
• Error stacks that are "usable" in optimized/LTOed builds
• Super aggressive use of const generics (eg. Vec has a fixed size variant that's API compatible with dynamic sized)