Change the default value of similarityCutoff parameter

[tomasonjo]

Not sure if there is a deeper reason for similarityCutoff to have to be greater than 0.0 to allow writeback. Fixes #729

[tomasonjo]

I checked other similarities and they all have similarityCutoff to have to be greater than 0.0, but the default is -1, so they don't writeback by default.

We either have to change the default value of similarityCutoff or remove this condition when considering if we should writeback or not on others aswell.

I would just remove the condition.

[mneedham]

The extra condition is deliberate as we thought it doesn't make sense to write a SIMILAR relationship between two nodes if they aren't actually similar in any way.

I think we should change the default to be -1 for all of them, but then update the documentation so that it's clearer what you have to do to write relationships.

I'm not really opposed to what you're suggesting either though so I guess @jexp can be the tie breaker!

[tomasonjo]

That would make sense, but the rule should be imposed one layer down. As is right now no relationships get written back even if they have similarity greater than 0 if we don't define similarityCutoff. So that rels with similarity greater than 0 still get written back.

And this only plays role in the context of cosine similarity as it is the only one that can have negative values, where jaccard and euclidian are always greater than or equal to 0 anyway.

[jexp]

I'm not sure 100% sure what you mean.

We should never write relationships for dissimilar pairs as it just explodes with n^2.
I want the defaults to be safe, i.e. if you don't specify something it should not hurt you.

The rule is different depending on the type of similarity.
So we probably only have to fix the default for cosine or make it by similarity != default.

[tomasonjo]

What I would suggest then is the following:

We got four similarities with possible values:

• cosine: -1 to 1
• jaccard: 0 to 1
• overlap: 0 to 1
• euclidian: 0 to ∞

If we keep it simple and just use one default similarityCutoff I would use 0.5 as cosine can still blow up with 0.0 still and we cover jaccard and overlap as they can blow up too.

On the other hand it is very hard to set an arbitrate default number for euclidian, so maybe we just stick with 0.5.

We can still include the boolean condition for cosine writeback to be greater than 0.0 and never allow user to writeback dissimilar pairs, but need to include it in the documentation then.

WDYT?

[tomasonjo]

Much cleaner option might be setting default topK parameter at 20 and not set any default similarityCutoff. This handles the default version very nicely IMO

[neo4j-oss-build]

If we don't set the cutoff the compute effort is exponentially high I don't think that's preferred. The default should for each algo to cut of the dissimilar nodes. Von meinem iPhone gesendet

…

Am 04.10.2018 um 19:10 schrieb Tomaz Bratanic ***@***.***>: Much cleaner option might be setting default topK parameter at 20 and not set any default similarityCutoff. This handles the default version very nicely IMO — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread. -- You received this message because you are subscribed to the Google Groups "neo4j-oss-build" group. To unsubscribe from this group and stop receiving emails from it, send an email to ***@***.*** To post to this group, send email to ***@***.*** To view this discussion on the web visit https://groups.google.com/a/neotechnology.com/d/msgid/neo4j-oss-build/neo4j-contrib/neo4j-graph-algorithms/pull/730/c427097653%40github.com.

[tomasonjo]

I changed the default similarityCutoff to 0.1. This implies to all except for euclidian distance where I left the default value -1.0.

[mneedham]

I think what we'll do is throw an error if you try and set write: true with a similarity of <= 0.0. I'll make a PR to do that and cc you on it