Skip to content

Commit

Permalink
improve performance for slice zeroization (issue RustCrypto#743)
Browse files Browse the repository at this point in the history
The purpose of the change is to make calls to `x.as_mut_slice().zeroize()`
considerably faster, particularly for types like `[u8; n]`.

The reason it becomes faster is that the call to `volatile_set` before
this change appears not to be easily optimizable, and (for example) leads
to setting bytes one at a time, instead of the compiler consolidating them
into SIMD instructions.

In the modified code, we don't use `volatile_set`, we instead loop over
the slice setting the elements to `Default::default()`, and to ensure
that the writes are not optimized out, we use an empty asm block.
There is discussion of the correct asm options to use here in the issue.

Because the asm block potentially reads from the pointer and makes a
syscall of some kind, the compiler cannot optimize out the zeroizing,
or it could cause observable side-effects. In the improved code, we only
create such an optimization barrier once, rather than after each byte that
it is written.

The call to `atomic_fence()` is not changed.

---

This change may help give users a way to improve performance, if they
have to zeroize very large objects, or, frequently have to zeroize many
small objects. We tested code-gen here in godbolt (in addition to the
tests posted in the github issue) and found that this change is
typically enough for llvm to start adding in SIMD optimizations that
zero many bytes at once.
  • Loading branch information
cbeck88 committed Feb 28, 2023
1 parent 0256fff commit 9168c06
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion zeroize/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -471,7 +471,17 @@ where
// object for at least `self.len()` elements of type `Z`.
// `self.len()` is also not larger than an `isize`, because of the assertion above.
// The memory of the slice should not wrap around the address space.
unsafe { volatile_set(self.as_mut_ptr(), Z::default(), self.len()) };
for z in self.iter_mut() {
*z = Z::default();
}
unsafe {
core::arch::asm!(
"/* {ptr} */",
ptr = in(reg) self.as_mut_ptr(),
options(nostack, readonly, preserves_flags),
);
}

atomic_fence();
}
}
Expand Down

0 comments on commit 9168c06

Please sign in to comment.