Process transfers one at a time #644
Description
Is your feature request related to a problem? Please describe.
The TransferRoutine currently processes all waiting transfers at once. If a lot of transfers are waiting this means they must all be moved and unzipped before any of them can be validated and saved. This is inefficient and error prone and means that if the disk fills up, the entire process is blocked.
Describe the solution you'd like
Make the DiscoverTransfers cron discover a single transfer at a time. This will require changes to move the for loop out of the TransferRoutine, and instead to pass in a bag to process.
Describe alternatives you've considered
This is not technically broken right now, but it could be better.