Moolib Backend Issues

[xiaomengy]

Recently there are several issues from moolib backend.

1. Based the observation of Performance regression after PR#26 moolib#36, there is a performance regression in moolib.
2. There are several installation issues in moolib.

Based on this we are thinking about building another backend not using moolib. Open this issue to track the progress.

PR for gRPC backend: #63

[xiaomengy]

After facebookresearch/moolib#38 moolib fixed the performance issue by adding a new non TensorPipe backend. Based on this we can keep on using moolib and #68 will fix that.

[d3sm0]

Why not using the native torch rpc backend instead? Is there a comparison about the speed improvement between torch rpc and moolib?

[xiaomengy]

The default backend of TorchRPC is also TensorPipe. The issue of TensorPipe is it has a large overhead especially when there are a large number of connections. So actually in the latest version of Moolib we have a new backend. But I can try to compare them to provide some numbers here if you are interested.

[d3sm0]

Yes please, if it's not too much of an hustle. it would be interesting to see what is the trade-off of a custom rpc library, and how much of this performance gain come from the dynamic batching that one could implement on torch rpc. I also saw a branch with grpc. What is the speedup from moolib compared to grpc?