In RNNs, gradients accumulate over time steps. If the sequence is long, gradients can become very large (exploding gradients) or very small (vanishing gradients), leading to unstable training or difficulty in convergence. Detaching the hidden state can limit gradient propagation within each time step, preventing gradient accumulation over the entire sequence, thus mitigating exploding/vanishing gradient problems.
In RNNs, gradients accumulate over time steps. If the sequence is long, gradients can become very large (exploding gradients) or very small (vanishing gradients), leading to unstable training or difficulty in convergence. Detaching the hidden state can limit gradient propagation within each time step, preventing gradient accumulation over the entire sequence, thus mitigating exploding/vanishing gradient problems.