Distributed dataparallel implementation of WaveRNN training

[yangarbiter]

Some speed up over the original WaveRNN training example.

Here is a working version on gist.

On a 8 GPU machine, it should be 10 times faster with batch size 256.

[nateanl]

Thanks for the proposal! I think another refactoring direction is using PyTorch Lightning template besides the DistributedDataParallel dataloader. We can put the two refactor way into one PR.

[yangarbiter]

One thing for this gist that I would like to add is that there are some command line arguments that are not well set.
For example, here and here, the default is True and using the argument also sets the variable to True, which is weird and should be fixed during the refactoring.