Tasking support

[krasow]

#66 originally presented these changes; however, the implementation has diverged significantly since then. #66 was closed, and all of its changes are now incorporated here, updated to support the latest Legate.jl releases.

#96 provided envs that changes the interaction of Julia and Realms use of SIGSEGV. This was a fundamental blocker in #66 as tasking code would error w/ a segfault. Due to leveraging these envs, arbitrary Julia code has the ability to be stable.

For correctness across multiple partitions, we added both device and host strided arrays- providing pointer indexing math to get the correct partition for the correct thread.

Status:

• CPU developer tasking CI passes across Julia 1.10–1.13.
• GPU Buildkite CI is not currently enabled for Legate.
• On a single NVIDIA A30X, a mixed CPU/GPU stress test passes with GC enabled.
• A CPU-only stress test runs in CI and passes across Julia 1.10–1.13.
• Experimental tasking support works with under and over specification. However, at minimum, there needs to be 1 default thread and 1 interactive thread. This is the default configuration of Julia 1.12 and 1.13. This is not the default configuration on 1.10 and 1.11. We have provided guards on enabling experimental tasking to error on invalid thread configurations.
• Merged Bloat constraint for halo exchange boundaries #107 which enabled bloat and broadcast constraints for tasking.

TODO:

• Test under- and over-specification of Julia threads relative to Realm processors.
• Test GPU tasking on 1.10, 1.11, 1.13. I have only tested locally on 1.12.
• Test multi-GPU tasking.
• Pass stress test with GC.enable(true)

Debugging and commits here were assisted by Claude Opus.