Enhancement: merger should upload blocks in background and work on next block #78
Description
copy from: streamingfast/merger#26
uploading blocks to s3 is sometimes slow. merger appears to do nothing until the merged block is uploaded, and then merges the next block.
merger would run much faster if it merged the next block while the previous block was still uploading. ie there should be an upload queue of merged blocks.
This issue occurs when you have a lot (1000s) of one-block files accumulated and merger can run at 100% CPU.
Workaround for this issue: have merger write files to local disk and manually send them to s3 later.