Support for FCS3.2

[JcGKitten]

Hey @whitews,
there is the new FCS3.2 standard which has some new keywords, especially
the PnDATATYPE. This allows single columns to have another datatype then the one set in DATATYPE keyword. See 3.3.41 on page 41. Do you already have plans to support that? It might be a problem cause the values in the used array must be of the same type as I understand it.

I will also think about a solution but wanted to check in with you first.

Best wishes
Max

[whitews]

Hey Max,

Yes, I've known about the new 3.2 version and read through the spec doc. Agreed, the mixed datatype will require an overhaul to the FlowIO code. The issue is I haven't been able to find any example 3.2 files. I think there's only 1 BD cytometer that exports these files, but could be wrong about that. In any case, we need one or more example files, specifically ones that are public & showcase the new features so we can add them to the repo with test cases. Ideally, it would be nice if they weren't ginormous. Do you have access to any that you can share?

Regards,
Scott

[JcGKitten]

Hey Scott,
I got a few files from a BD S8 cytometer, but I have to check back if they can be made public and I'm not sure if they showcase all the new stuff. Also they have some values, like range limits, which seem very odd to me. I let you know if I can provide some samples. Thanks for letting me know that you are aware of the new standard.

[wiegman-miftek]

I'd like to use 3.2 features with FlowIO as well. Is there a roadmap to making this happen yet @whitews? I'd be happy to contribute some code if you can tell me what to work on.

[whitews]

Hi TJ (@wiegman-miftek),

Thanks for offering to help, I've been busy on other projects so any help would be appreciated. The first step would be collecting a few FCS 3.2 files that we can include in the repository. Those files would need to include the new features in the 3.2 specification. Do you have files that can be shared publicly?

After that we can discuss the next steps in modifying the FlowIO code base.

Kind regards,
Scott

[wiegman-miftek]

I asked around my lab and unfortunately we don't have any commercial machines that natively output 3.2 files. I suspect that if we want to add 3.2 support then we'll just have to write a novel implementation strictly according to the published spec.

In the meantime, I think users can "spoof" most of the new features from 3.2 (and then manually mark the output file as 3.2) with the existing release of FlowIO, by manually writing the appropriate data to the newly added standard key-value pairs. The only major feature that can't be mimicked this way would be the new per-channel data type system (e.g. int for time channel, float32 for spectrometry channels, float64 for another channel, etc), since FlowIO currently serializes all channels with the same method.