Disk caches should only write on some ranks

[connorjward]

The way it is currently implemented, DictLikeDiskAccess writes to disk on all ranks. This is clearly inefficient. Instead we should try and use something like the compilation_comm (created here) and only write on one of the ranks that share a filesystem.

My idea for how to approach this is to modify DictLikeDiskAccess with a dummy: bool kwarg s.t. access on particular ranks can be masked. For example:

def __setitem__(self, key, value):
    if self.dummy:
        pass
    else:
        # actually set the value

This attribute could be set doing something like:

def _make_disk_cache(comm):
    filesystem_comm = comm.getAttr("filesystem_comm")
    dummy = filesystem_comm.rank != 0
    return DictLikeDiskAccess(cachedir, dummy=dummy)

@disk_only_cache(cache_factory=_make_disk_cache):
def cached_func(...):
    ...