Espressif 20Mhz operation breaks usb operation till next reset

[bill88t]

CircuitPython version

Any > #9342, discovered on S3

Code/REPL

import microcontroller
microcontroller.cpu.frequency = 20_000_000 # 20MHz

Behavior

Usb goes away. This is fine as by itself, as usb cannot operate with the board being at 20MHz.
However when switched back to a higher clock, usb does not start back up, even if the board is unplugged and plugged back in (assuming it's battery powered).

[bill88t]

This does not happen if the USB filesystem is disabled from boot.py.
In other words, USB reconnects when just USB-CDC is enabled.