Provide way to catch OOM in C API

[seanmonstar]

Currently, allocation errors automatically result in an abort, which is undesirable in the C API. We could look into using std::alloc::set_alloc_error_hook or similar, to override the default behavior so that it doesn't abort, but does some sort of panic!(HyperOom), and we catch that at the FFI boundary (like all panics). However, that is not a stable function. First step is likely to get libstd to stabilize something we can use.

Originally from #2265 (comment)

[seanmonstar]

As a check-in (at least for myself), there's the following tracking issues for the lang/libstd:

• Tracking Issue for OOM hook (Tracking issue for the OOM hook (alloc_error_hook) rust-lang/rust#51245)
• Tracking issue for oom=panic (Tracking issue for oom=panic (RFC 2116) rust-lang/rust#43596)

The status of those issues don't seem to have changed. It may need more active attention from us.