macOS dmg comments

[tschoonj]

Hi,

I stumbled onto your project and decided to give it a go but ran into a couple of issues. I hope you will find this useful.

The DMG image contains a single folder called Contents. This folder is on its own unusable as it needs to be put in a folder called Peakaboo.app. Afterwards Peakaboo starts as expected.

One other problem I noticed is that it looks horrible on retina HIDPI screens (like mine). This could be fixed by adding:

<key>NSHighResolutionCapable</key>
<true/>

to Info.plist.

Afterwards things look much better.

However I still couldn't really test your app as it doesn't support HDF5/Nexus files 😢
Any chance you could add support for these file formats? We have plenty of java code for it in DAWN...

Best,

Tom

[nsherry4]

Hi,

Thanks for the feedback. I don't have a Mac to test Peakaboo on, so I'm never completely sure that a release will work properly on it.

I think I've implemented the changes you suggested, and I've created a new test build here: http://nathaniel-desktop.beowulf.uwo.ca/Peakaboo/Peakaboo%205.1.0.dmg. Let me know if those changes resolve the issues you're having and I'll update the releases page.

We can absolutely look into adding support for the HDF5/Nexus files; we already have some other file formats supported which use HDF5 containers.

On Peakaboo's end, the code would need to implement the DataSource interface. There's other code designed to make implementing this interface easier, but this is the core of it. For a more practical example, the PlainText plugin is how Peakaboo reads CSV files.

Any chance you could point me at the HDF5/Nexus source on your end?

[tschoonj]

Hello again,

The new app bundle is a lot better: thanks for that

Our hdf5 plugin resides at https://github.com/DawnScience/dawn-hdf. It really just wraps the official HDF5 Java bindings in an eclipse plugin, and includes the platform dependent HDF5 shared libraries.

The Nexus reader code is at https://gerrit.diamond.ac.uk/plugins/gitiles/scisoft/dawnsci/+/master/org.eclipse.dawnsci.nexus/ (no GitHub mirror at the moment). This one will drag in a lot of other plugins though.

I noticed that you are using a modified version of my xraylib jar: if you think that your changes are useful for the official release, feel free to submit a pull request to include them.

Best,

Tom

[nsherry4]

I've updated the releases page with the improved app bundle.

Any suggestions for the best way to start going through the Nexus code? I'm not sure if I should be looking at the autogen code or not, for example.

If linking the software is going to take a bit, and if the file format isn't too complicated, maybe a good first step would be to just write something quick that can handle the basics. Do you have any sample data that you're able to send to me?

I don't think I made any changes to Xraylib, but I couldn't find it in maven, so I bundled it in a little local repository.

[tschoonj]

I've updated the releases page with the improved app bundle.

Cool! Thanks

Any suggestions for the best way to start going through the Nexus code? I'm not sure if I should be looking at the autogen code or not, for example.

Hmm you may want to have a look at org.eclipse.dawnsci.hdf5.nexus.NexusFileHDF5 to open a file for reading. Not sure how to list the groups and datasets in the returned object though... @PeterC-DLS @jacobfilik Thoughts?

If linking the software is going to take a bit, and if the file format isn't too complicated, maybe a good first step would be to just write something quick that can handle the basics. Do you have any sample data that you're able to send to me?

I will ask around for a file I can give you for testing.

I don't think I made any changes to Xraylib, but I couldn't find it in maven, so I bundled it in a little local repository.

Ok. So what would I need to add to make it acceptable for maven? Just a pom.xml file?

[PeterC-DLS]

There are two ways to open a Nexus file:

1. Use theloadTree() method from HDF5Loader or NexusHDF5Loader in ukac.diamond.scisoft.analysis. This loads the entire tree.

2. Create an org.eclipse.dawnsci.nexus.NexusFile instance via INexusFileFactory (using OSGi, see example in org.dawnsci.conversion.converters.Convert1DtoND). Open to read, then navigate using getGroup(). Note this lazily populates the tree from file so you need to call getGroup as you get deeper.

[nsherry4]

Adding the pom.xml file would let you build the project with maven, but actually uploading code to the maven central repository is something I've never explored myself.

I'm still trying to figure out how to properly build the code. I've downloaded a copy of:
https://gerrit.diamond.ac.uk/plugins/gitiles/scisoft/dawnsci/+/master
as well as other repositories org.eclipse.richbeans and dawn-hdf listed in the travis build file, but I still get errors about missing parent pom:

[nathaniel@nathaniel-desktop dawnsci]$ mvn build
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.eclipse:hdf.hdf5lib:1.10.2-SNAPSHOT (/home/nathaniel/Projects/SS/Git/DLS/dawn-hdf/hdf.hdf5lib/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM: Could not find artifact org.eclipse:org.eclipse.dawnsci:pom:1.0.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 7, column 10 -> [Help 2]

Any documentation (or a pre-built jar)?

[tschoonj]

Based on the travis.yml file, it would seem that you need to rename the dawnsci directory to org.eclipse.dawnsci.

If you would like to get your hands on pre-built jars, I recommend you download the latest version of DAWN from https://dawnsci.org

[nsherry4]

I made some progress with the maven build, but ran into problems later on.

[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: org.eclipse.dawnsci.analysis.api 1.2.0.qualifier
[ERROR]   Missing requirement: org.eclipse.dawnsci.analysis.api 1.2.0.qualifier requires 'bundle org.eclipse.january 2.1.5' but it could not be found

I got the jars from DAWN, and that seems to be working for now. At least, my IDE shows no errors.

Am I correct in understanding that a Nexus files can contain more than one dataset? If so, I think the next step is to figure out how to get a list of those datasets for the user to choose from. I see a way to iterate over attributes, but not nodes.

[PeterC-DLS]

HDF5 is like a file system inside a file with groups acting like directories. Read the fine Nexus user manual

[tschoonj]

The January source is hosted here in case you would need it.

Nexus files can indeed contain multiple datasets, and they often do here at Diamond Light Source.

To iterate over nodes, you will need to start with the root. Something like this:

NexusFile file = NexusFileHDF5.openNexusFile("yourNexusFile.nxs");
String rootName = file.getRoot();
GroupNode rootNode = file.getGroup(rootName, false);
List<GroupNode> nodes = rootNode.getGroupNodes();

[nsherry4]

I was able to find a random .nsx sample file here:
https://github.com/DawnScience/scisoft-core/blob/master/uk.ac.diamond.scisoft.deprecated/testfiles/gda/analysis/io/NexusLoaderTest/NexusLoaderTest.nxs

From looking at it in HDFView, It looks like the schema being used here is /scan#/dataset#/data. If that's the schema in use here, too, I'm thinking the user should be asked to choose between each /scan#/dataset# when opening the data. Does that make sense?

[nsherry4]

@tschoonj I think I've done about all I can without a proper sample file to look at. Any luck finding one?

[tschoonj]

I have asked around for one but no response yet... I'll ask some other people...

Either way, the /scan#/dataset#/data structure you mentioned before is not going to be something you can rely on.

Usually there will be one or more NXentry groups put at the root level. Each of these NXentry groups will then in turn contain several NXdata groups. Such groups should contain a single principal dataset (referred to as the signal), possibly along with its associated axes datasets. If this is a properly structured NXS file (and this is a really big if as such files are a rare and precious thing...), there should be attributes in the NXdata group that associate the axes datasets with the corresponding dimension of the signal dataset.

Hope this helps...

[nsherry4]

I'll also add that, until we get some good sample data to test this plugin with, there is one other option for trying out Peakaboo. If you can convert an existing dataset to a csv/tsv file with one spectrum per row, Peakaboo will be able to open that. It loses all the meta-data, but it gets the data imported.

[tschoonj]

I have asked 3 of our principal beamline scientists but no reply...

Thanks for the CSV suggestion: I will give it a try but I am kind of busy at the moment...