Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

now reading from that file, dataset dimensions seem to be mismatching #28

Open
clive-g-brown opened this issue Jul 31, 2019 · 3 comments

Comments

@clive-g-brown
Copy link

Error: unhandled exception: Wrong input shape of buffer to write to in read. Buffer shape @[1497098, 2000], dataset has shape @[1497098, 2000] [ValueError]

@Vindaar
Copy link
Owner

Vindaar commented Jul 31, 2019

Sorry, you're stumbling over these things probably due to my lack of documentation.

While that read proc is exported, it's probably not what you want to use.
I assume you want to read a whole dataset?

For that you should either use readAs to read (and potentially convert the data type) either from a H5DataSet object you already have, or the H5FileObj and the path to the dataset you wish to read, like either:

let data = dset.readAs(<some datatype>)

or from the H5 file

let dsetName = "somePath/to/a/dset"
let data = h5f.readAs(dsetName, <some datatype>)

Note, that readAs will first read the data of the dataset according to the data type that it has in the H5 file. To return the data to you, it will be converted simply via T(val). So be careful. Converting a float to an integer won't use any rounding for instance. Also the conversion must make sense.

Alternatively using the [] accessor procs like either:

# from the dataset object you might have
let data = dset[<the datatype of the dataset>] 

The data type is required, since the [] proc is generic and and Nim doesn't allow overloading by return type.
Or from the h5 file:

let dsetName = "somePath/to/a/dset"
let data = h5f[dsetName, <the datatype of the dset>]

In general look at the tests (for lack of tutorials and documentation):

@clive-g-brown
Copy link
Author

thanks I couldn't get readAs to work

Error: type mismatch: got
but expected one of:
template .()(obj: CppObject; field: untyped; args: varargs[CppProxy, cppFromAst]): CppProxy
../../anaconda3/envs/nimtorch/dist/pkgs/fragments-#head/fragments/deps/nimline/nimline.nim(29, 5) CppObject: concept predicate failed

however the [] works.

@Vindaar
Copy link
Owner

Vindaar commented Aug 1, 2019

Hm, the error you get when using readAs looks like a result of failing overload resolution (or rather nim not finding the readAs symbol for the way you called it).
Since nimline seems to define that template, which absent of some other proc that takes precedent over an untyped template, will match any . call of some CppObject and any field. Therefore instead of an undeclared identifier: readAs error, you see the one above.

The likely reason for this I assume is that you're using version 0.2.13 of nimhdf5 instead of the #head of master. I haven't pushed a new version in a while and readAs as a convenience proc was only added recently.
If you are on #head and still see the issue, I suppose one of the types in your code didn't match. If you have the code somewhere to look at I could comment on that.

I'll push a new version!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants