Is there a way i can ignore scale while calculating consensus motifs?

[0tist]

Hi, I have a graph which is monotonically increasing. I wish to compare the sub-sequences to determine how it's growing, but i want to ignore the scale, as in i want to compare the sub-sequences such that they are in the same range, both the query and the reference sub-sequence

[JaKasb]

You can disable the individual z-Normalization of the subsequences with the parameter normalize=False.
https://stumpy.readthedocs.io/en/latest/api.html#ostinato
Be aware that this disables both normalization methods. 1. Shift to zero mean. 2. scale to unit variance
If you want to disable scaling but want to keep the shift, then you are out of options.

In your case, try to detrend your timeseries beforehand and then run stump with normalize=False.

[0tist]

thanks for your response, I dont understand what do u mean by 'detrend'?

[JaKasb]

You fit (linear) model to your data.
Then compute data(x) - model(x)
Thereby removing any trends, which are modelled in the fitted model.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.detrend.html

[seanlaw]

@0tist If I understand your question correctly, it sounds like you have multiple time series with different magnitudes but that you suspect that the shape of the subsequence(s) might be similar (even if one subsequence is scaled up, say, 5x as large). By default, STUMPY applies a "z-normalization", which essentially helps to ignore the differences in magnitude between different subsequences. In other words, the scale of each subsequence is normalized first before the distance is computed between the subsequences. However, if you set normalize=False, then the subsequences are NOT normalized and STUMPY computes the regular Euclidean distance between the raw subsequences (i.e., if there are large differences in the magnitude between the subsequences then the Euclidean distance will be correspondingly large). I hope that helps.

[0tist]

Yeah, thank you for the crystal clear response. I understand it now, but when I plotted the consensus motifs, I could see the original scales of the motifs, why is that so?

[seanlaw]

@0tist So, since stumpy.ostinato only returns three values:

1. bsf_radius (the "best-so-far" radius)
2. bsf_Ts_idx (the "best-so-far" time series index)
3. bsf_subseq_idx (the "best-so-far") subsequence index

One then needs to take this information and then search across all of the time series to find relevant matches (see this example) to this consensus motif:

seed_motif = Ts[Ts_idx][subseq_idx : subseq_idx + m]
x = np.linspace(0,1,50)
nn = np.zeros(len(Ts), dtype=np.int64)
nn[Ts_idx] = subseq_idx
for i, e in enumerate(Ts):
    if i != Ts_idx:
        nn[i] = np.argmin(stumpy.core.mass(seed_motif, e))
        lw = 1
        label = None
    else:
        lw = 4
        label = 'Seed Motif'
    plt.plot(x, e[nn[i]:nn[i]+m], lw=lw, label=label)
plt.title('The Consensus Motif')
plt.xlabel('Time (s)')
plt.legend()
plt.show()

Since this example uses nn[i] = np.argmin(stumpy.core.mass(seed_motif, e)) (i.e., computes z-normalized distances in order to find the index location of the matches) and then returns the index location of "one nearest-neighbor" for each time series, then I think that this example should have also z-normalized the subsequences before performing the plotting. Otherwise, the original raw (non-normalized) subsequences from each time series will be plotted instead. So, replace the line:

plt.plot(x, e[nn[i]:nn[i]+m], lw=lw, label=label)

with its z-normalized subsequence equivalent:

from stumpy.core import z_norm
plt.plot(x, z_norm(e[nn[i]:nn[i]+m]), lw=lw, label=label)

Please let me know if that works. Perhaps, this procedure should be highlighted better in the tutorial?

[0tist]

Thank you so much for your response, it has helped me a great deal!
It would be better if you highlight that in the tutorial!
Thanks again :) 👍🏼

[seanlaw]

@0tist I've gone through and greatly simplified the code in order to focus on the core concepts and to be more "accurate" for extracting z-normalized subsequences. Note that your browser may cache some of images displayed in the tutorial so it may take some time for them to be refreshed accordingly (but the code should be new). Hopefully, the code is more accurate now.

https://stumpy.readthedocs.io/en/latest/Tutorial_Consensus_Motif.html