Use sd_vector_builder during colour matrix construction

[alexbowe]

sd_vector_builder allows appending of 1-bit positions. This approach will save us from having an intermediate uncompressed bit matrix.