Suggested first steps

[jorainer]

Dear @chufz @michaelwitting @hechth,

I would suggest to first collect all the code and examples you already generated and put that into the README (maybe in separate subsections?).

Would also be cool if someone could add installation requirements there.

Also, I to avoid conflicts etc, I suggest everyone works in her/his own branch (I created already some) and merge this from time to time to the main branch.

Generally, I think this will be a very cool project. I always wanted to combine our code with the really great software that is available in python. So, thanks for kick-starting it!

[michaelwitting]

As first step the documentation of matchms:
https://matchms.readthedocs.io/en/latest/

And the GitHub repo:
https://github.com/matchms/matchms

[michaelwitting]

A first required function would be the conversion of R Spectra into Python Spectrum. While in R Spectra can handle multiple spectra, Python Spectrum contains only a single spectrum. Multipe spectra are stored in [].

[michaelwitting]

My second thought is how we want to make comparison function from matchms accessible for the compareSpectra function or do we implement new functions?

[jorainer]

Maybe we split some more detailed discussion into different issues (e.g. #3 for the Spectra -> Spectrum).

[jorainer]

The second question is a good one. Maybe we could first try to convert Spectra to matchms::Spectrum, do then the similarity calculations in python (maybe they are faster there? loops will definitely be faster in python) and return the results.

In a second approach we could try the other way round, do all in R but calling the spectra similarity function from matchms (i.e. within the compareSpectra calls, convert to python, calculate, get results). That way we could benefit from Spectra to not needing to have all data in memory, but it might be slower...

[hechth]

Interfacing matchms this way is definitely possible, but with the MSP backend, these two packages should already be compatible in their native environments.

[chufz]

@hechth, yes this would also be possible, but inconvenient for the R programmer, and when your using a complete workflow in R. As well as msmatch does not store the metadata so efficiently as in a spectra object. But it would be a convenient way for me if I want to compare a large dataset.

My idea would be to have it in the compareSpectra function so that the spectra similarity algorithms of msmatch can be also used in the MetaboAnnotation or any other comparison workflow. @jorainer is this possible or we might need a similar function compareSpectripy which is producing the same output, but with the different similarity measure.

[michaelwitting]

The problem is that compareSpectra uses a two-step approach. Two different functions are supplied to compareSpectra, MAPFUN and FUN. The first one aligns the two spectra and returns the aligned spectra to FUN for the actual score calculation. Functions in matchms do everything in one step.
I have the feeling we need something like compareSpectripy.

[jorainer]

No objections against a different function - but IMHO the loop and comparising should all be performed in Python (if possible). The worst would be to have a for loop or lapply in R that iterates over a Spectra and in each iteration converts the peak data to python, performs the similarity calculation and converts the result back.

Better in my opinion: convert a whole Spectra to python, perform all calculations there, retrieve the results and convert back to R. But I have no idea if that would work - and also, I'm just guessing this might be faster/better, we would need to formally compare the approaches to see what's better.

[michaelwitting]

I think we can write a nice wrapper around calculate_scores from matchms. IMHO all functions that we write from comparison shall accept R objects and return R objects. Handling of Python things and conversion shall be handled within functions. Just to make it as similar as possible to Spectra

[jorainer]

I would suggest the following:

• I'll have a look into how to build the package so that it is easiest for future users (installation of requirements etc all handled by the package if possible).
• @michaelwitting (you already did that): implement the functions to convert Spectra to and from python
• @chufz (with help from @hechth ?): implement a first function in python that loops over two arrays of python spectra objects calculates the similarities between each pairwise elements and extracts the scores.

The first use case could thus be to have an R function that 1) converts the Spectra to an python array, 2) calls a python script to calculate the pairwise similarities and 3) converts the results back into a n x m matrix which is returned to the user.

[michaelwitting]

I will add the documentation and subselection of the metadata to it.
At the moment it is returning a list of Spectrum, which is a mixture of R and Python. I would change to code in such a way that directly an array for Python is returned.
@hechth : compare_scores only accepts arrays, right? Even if it is a single spectrum, it shall be an array of length 1?

[michaelwitting]

@chufz and @hechth I think we can use the compare_scores directly in R with reticulate? Like this we don't have to write another loop.

[chufz]

I would suggest for simplicity to have one phyton script that is executed with py_run_file. Should be possible or?

[michaelwitting]

We can check, what is faster: Running it with reticulate or directly in Python. I will change then the return of the conversion directly to a Python object, then we don't have to think about including r_to_py all the time.