How the ties are handled?

[cyuab]

For a given time series $T$, we can compute the Matrix Profile of $T$ by

mp = stumpy.stump(T, m=m)

And access the nearest neighbor and the corresponding distance of all the m-subsequences (i.e., length = $m$) by

mp.I_
mp.P_

So, if for a subsequence $T'$ at the position T_prime_idx, there are two nearest neighbors in $T$ that have the exact same distances from $T'$ (i.e., mp.P_[T_prime_idx]), which one will be picked for mp.I_[T_prime_idx]?
Any particular preferences? For example, the one closer to $T'$.
Sorry if it is written in the document.

[seanlaw]

Good question. There is no preference. Whichever nearest neighbor that was encountered first during the distance computation will be kept. Note that since this is done in parallel, it is not known what the order will be.

[cyuab]

Thanks for your reply. I agree it is challenging to know which one has been kept.
In some scenarios, I think it is useful to have a preference to handle the ties. For example, always keep the one that is closer to $T'$.

[seanlaw]

I think you mean matrix profile instead of distance profile to find out the top-2 smallest distances, right?

No, I mean "distance profile". Since you were asking about ties and whether or not the one-nearest-neighbor is "closer" (by index location), you could verify this using the approach above by computing the distance profile for a subset of subsequences that were identified as having a one-nearest-neighbor below a desired distance threshold. Each of these identified subsequences would act as the query subsequence and you would compare it to all subsequences to its left, stumpy.mass(T[start_idx : stop_idx], T[ : start_idx - excl_zone]). Notice that we can save some computation by truncating the time series. Then, you can compare the values in the distance profile with the value computed from the left matrix profile. If there is more than one "smallest" distance then you may need to update your nearest neighbor location.

[cyuab]

Thank you so much again. This idea is neat!

[seanlaw]

I am finding the motif of a given subsequence in gene expression data. I want to find the ones that are earlier than the given subsequence. If there are ties, I would like to retrieve the closer one.

Interesting. Given that genes aren't continuous (floating point) values, does this mean that you are somehow able to encode your nucleotides as a numeric value? If so, how do you calculate a "distance" with these values? In case it matters, I use to be a biochemist (a long, long time ago but I now work in finance) and so I've always wondered how you'd effectively compute a "Euclidean distance" for genes (any reference to papers or best practices would be welcome).

[cyuab]

For the time series data related to genes, I think there are two types of them.
One is the time series encoding for the string of DNA/RNA [1][2].

The other one is something called gene expression data.
When we measure the gene expression profile of a cell, we kill it.
So it is impossible to keep track of the gene expression profile dynamics of a single cell.
In about 2014, a computational technique called "trajectory inference" emerged, which could construct a trajectory of the cells [3].
[4] is a good website for the introduction of "trajectory inference". Please read the section "Inferring trajectories" to know more about it.

References
[1] W. Yan et al., “Time series-based hybrid ensemble learning model with multivariate multidimensional feature coding for DNA methylation prediction,” BMC Genomics, vol. 24, no. 1, p. 758, Dec. 2023, doi: 10.1186/s12864-023-09866-5.
[2] T. Zhang, Q. Tang, F. Nie, Q. Zhao, and W. Chen, “DeepLncPro: an interpretable convolutional neural network model for identifying long non-coding RNA promoters,” Briefings in Bioinformatics, vol. 23, no. 6, p. bbac447, Nov. 2022, doi: 10.1093/bib/bbac447.
[3] C. Trapnell et al., “The dynamics and regulators of cell fate decisions are revealed by pseudotemporal ordering of single cells,” Nat Biotechnol, vol. 32, no. 4, pp. 381–386, Apr. 2014, doi: 10.1038/nbt.2859.
[4] Single-cell best practices — Single-cell best practices

[seanlaw]

Thank you and best of luck with your research