Finding similar patterns with regards to all dimensions

[Yasaman-A]

For a time-series data that had n dimension, e.g., d_1, d2, …, d_n, we want to find out the similar patterns along all dimensions.
For example, for time series with 2 dimensions, even if two windows (t1 & t2) with regard to d_1 are identical, if for the same windows the patterns are very different with respect to d_2, we will not consider t1 & t2 as similar patterns. In another word, we want to find windows of size m that their total distance is minimum in all dimensions. Is there any efficient way with Stumpy to calculate this, or should we implement it ourselves using the distance matrixes?
Using mstump, we were able to find motifs in each dimension, but I believe each dimension is considered independent that other dimensions, right?

[seanlaw]

@Yasaman-A Thank you for your question and welcome to the STUMPY community. In case you may have missed it, I would strongly recommend going through the Multidimensional Motif Discovery Tutorial as it contains an in depth explanation of what a multi-dimensional matrix profile is and how it is different from computing n independent matrix profiles and then stacking them one on top of each other.

Using mstump, we were able to find motifs in each dimension, but I believe each dimension is considered independent that other dimensions, right?

So, each dimension of the multi-dimensional matrix profile is actually the average matrix profile value from the best possible subset of dimensions (i.e., a subspace). In your particular case, you'd want to compute the multi-dimensional matrix profile and use the values from the last row as that would mean that all dimensions are considered. As code:

import numpy as np
import stumpy

T = np.random.rand(40).reshape(4, 8)
m = 5
mp, indices = stumpy.mstump(T, m)

Here, since there are 4 dimensions, mp[3, :] (this is zero-based indexing) would give you the matrix profile that must include all 4 dimensions. Similarly, mp[0, :] is the matrix profile that must include 1 of 4 dimensions while mp[1, :] must include 2 of 4 possible dimensions. However, when you don't use the last dimension, you'll need to use some other STUMPY tools (i.e., stumpy.subspace and stumpy.mdl) in order to retrieve the subset of dimensions (i.e. subspace) that were involved in computing the matrix profile. However, this is a post-processing step since it is not possible store the subspaces on-the-fly. Please let me know if that makes sense.

[Yasaman-A]

Thanks @seanlaw for your quick reply and also for providing the resources.
You are absolutely right. The behavior of our data had made us think that the motif is selected independently. We were able to fix the issue and better understand how mstump is working. Thanks for the great library and for your help.

[seanlaw]

Great! Please feel free to post additional questions if you have them and I'll try my best to assist. Just a heads up that stumpy.mdl is a brand new feature and is only available on the development branch (and will be available in the next STUMPY release) so you'll have to install it from the Github repo if you want the latest feature.