Consensual dataset combination function is changing underlying dataset

[JoeRibeiro]

A different - worse - dataframe is presented after the dataframe is rebuilt to gain a consensus across labellers. Something is quite broken in the function compute_consensual_labels_and_sample_weights() - currently commented out but central to the training side of the tool.

I am unsure if the issue is "new", but previously models were training to high scores (~0.95), and now the score drops to ~0.6 if the function compute_consensual_labels_and_sample_weights is used. For now, this is just commented out.

[JoeRibeiro]

I may have been wrong to blame compute_consensual_labels_and_sample_weights() - even when commented, there is still a difference between
post_nn_cleaning_3d.html and premerge_3d_fluorescence.html

[JoeRibeiro]

Possibly fixed with commit 659f777
There were two things happening here:

Removing NAs caused a subtle difference between those two .html plots as axis limits were set differently

compute_consensual_labels_and_sample_weights() was also failing to recognise particle with id 1 from file 1 as different to particle 1 from file 2.

but now I need to test this with multiple labels.

Given further thought, I think it would be more robust to use sample datetime or a similar feature extracted from the metadata in the sample json rather than the filename. Filenames from different people being different and underlying data then being treated as separate is more likely to be an issue.