PR: Add average delta metric to probe_accuracy results

[garethky]

This adds a new metric called average delta that averages the delta between each pair or probes in the set. This metric is less susceptible to variance in the measurements conditions that happen over the set of probes, such a temperature changes and hysteresis of components.

With the load cell probe, its now very possible to see things like the bed changing shape as it heat soaks. This is not the assumption of PROBE_ACCURACY, all of its metrics assume that there is absolutely zero change in the environment between all probes. But in the real world this isn't true. So we see "bad" accuracy results which are not the fault of the probe itself.

average delta gives you a better idea of how repeatable your probe is, regardless of the underlying environmental changes. average delta tells you: "If I took 1 more probe, how different would it be from the last one". This is really what users are trying to get from PROBE_ACCURACY, as they use it to decide how many probes to perform.

Comparing average delta to range and standard deviation also gives you some idea about what was going on during a PROBE_ACCURACY run. If they all basically agree the environment was essentially static and the probe variance was random noise. If average delta is significantly better then its likely something was moving around during the test.

Checklist

• [✅] pr title makes sense
• [✅] added a test case if possible
• [✅] if new feature, added to the readme
• [✅] ci is happy and green

[dalegaard]

I think this is a good idea, but am wondering if the metric would be more useful highlighting bigger outliers more? So instead of doing absolute, doing root-mean-square?

[garethky]

I think this is a good idea, but am wondering if the metric would be more useful highlighting bigger outliers more? So instead of doing absolute, doing root-mean-square?

I don't know if anyone is going to understand what RMS means in this context. Actually I don't, so my guess in fewer than 20% of users would get that.

I picked "average delta" because I think its easy to understand. "average change between probes" is pretty intuitive concept.

---

About the only thing that would be easier to understand is a table of the number of probes and the resulting accuracy:

Probes  |  Accuracy (mm)
  1     |  0.12
  2     |  0.05
  3     |  0.025 <-- min suggested probes
  4     |  0.015
  5     |  0.01

This would have to make the assumption that the data being gathered is under adverse conditions (i.e. thermal drift), but the future sampling will be done under ideal conditions. Otherwise you'll get very different results with the same probe.

[dalegaard]

Makes sense. Since you are talking about thermal drift as well, maybe having both the mean absolute difference(like here) and the mean difference(so not absolute) would be useful? Mean difference would show if the position is stationary, and it's a well know test in statistics.