SKimage has morphology functions that can remove blobs.
from skimage import morphology
morphology.remove\_small\_objects(data, min\_size=32, out=data)
This approach seemed to work well on masks from sigma frames [(max - mean) / std] but may work better on masked argmax point clouds.