FlowKit imports FlowJo gates made on compensated channels with the uncompensated channels

[hbhargava7]

Describe the bug

If you create a gate on the compensated version of a channel in FlowJo (e.g. Comp-APC-A), and then get the gate via wsp.get_gate(sid, 'gate_name'), it uses the uncompensated version of the channels

Code To Reproduce
Code to reproduce the behavior:

import flowkit as fk
wsp = fk.Workspace(flowjo_path, find_fcs_files_from_wsp=True)
wsp.get_gate(sid, 'gate_name').get_dimension_ids()
>['APC-Cy7-A', 'APC-A']

but it should be

['Comp-APC-Cy7-A', 'Comp-APC-A']

get_gate_events returns events from the uncompensated channels too, regardless of whether source is passed as raw or comp.

get_gate_events also does not include the compensated channels, only the raw channels. However, wsp.get_transforms(sid) includes the compensated channels.

Expected behavior
The above code should return
['Comp-APC-Cy7-A', 'Comp-APC-A']

get_gate_events returns events from the uncompensated channels too, regardless of whether source is passed as raw or comp.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: macOS Tahoe 26.1
• Python version 3.11.13
• FlowKit version 1.2.3

Additional context
Add any other context about the problem here. Include traceback here

[whitews]

Hi Hersh,

In the parsing of .wsp files the FlowJo prefix for compensated channels is detected and used to pair any compensation matrix to the dimensions. However, the FJ-prefixed label for the channels is not used after that in FlowKit, in favor of keeping the labels consistent with the original FCS labels. You can see how this works in the wsp_utils.py module (search for "comp_prefix" or "Comp-" to find the relevant sections).

Hope that makes sense and let me know if I've misunderstood the issue. I do want to make sure you are referring to just the label names and not that you are seeing incorrect gate stats.

-Scott

[hbhargava7]

Hey @whitews – thanks for the quick response. I think I am indeed seeing the incorrect gate stats: the number of the events in the gate does not align with what FlowJo shows, by quite a large margin (e.g. 1592 events vs 839 events); less in FlowKit.

Though I am now realizing that this might not be due to a compensation issue – there is a small disparity for gates that don't involve compensation as well. However, this is much more egregious for some channels than others, perhaps actually because of negative intensity values (post compensation) getting mis-gated by Flowkit? (see attached)

Appreciate any insights!

Edit: here's where it's really problematic - same gate in wsp.plot_gate vs in flowjo

[hbhargava7]

ALSO - if I plot one rectangle gate on COMP-Channel and another on Channel-A and then visualize both with wsp.plot_gate, they both show the exact same underlying data distribution.

[whitews]

The gate stats can be different between FlowJo & FlowKit due to FlowJo binning the data, however this is generally negligible in terms of relative frequency. Can you increase the subsample count on the FlowKit plot so we can see a better comparison (the default is 10k events)? Also what is the absolute count and relative freq for that gate in FJ vs FK?

[hbhargava7]

Subsample count set to 1e7 (far excess):

in FJ: 62.2%, 3721 events
in FK: 1320 events in len(fx.wsp.get_gate_events(sid, 'CD4+ pSTAT5+')),
freq = 22.6% via len(fx.wsp.get_gate_events(sid, 'CD4+ pSTAT5+'))/len(fx.wsp.get_gate_events(sid, 'Single Cells'))

[hbhargava7]

Moreover -

Total number of events is the same in FJ and FK (by sample.get_events()) == 98036 events

        Live Cells (Zombie-): FK: 6629, FJ: 6649

                Single Cells: FK: 5847, FJ: 5987

                        CD4+ pSTAT5+ (the gate in question): FK: 1320, FJ: 3721

[whitews]

Ok, those parent gate differences are what I'd expect, FlowKit is usually under by a bit (again, due to the binning). But that CD4+ gate is too far off. I wonder if the binning issue is causing a large discrepancy because of the large number of "floored" events in the Alexa Fluor channel. Can you try duplicating the gate in FlowJo (so you can keep the original and get a copy) and nudging the boundary such that it is above those event on the edge?

[hbhargava7]

Yeah, excluding the floored events from the gate brings the count in FJ to ~1500, which is closer.
Do you think gates that extend past the chart boundary in FlowJo are getting parsed in wrong somehow?

[hbhargava7]

Yeah so the min value in the channel is 0.0 and the gate boundary is 0.0 (fx.wsp.get_gating_strategy(sid).get_gate('CD4+ pSTAT5+').dimensions[1].min)

Presumably the comparator (which I haven't found yet) is > not ≥ boundary_value, so the floored points get excluded.

What are your thoughts on what the expected behavior should be in this situation?

[whitews]

Yes, I think that is what is happening. FlowJo bins data in a grid (I think 256x256). All events in a grid square are then part of a "super event" (my made up term for lack of a better one). I'm not sure what coordinate location is then used for the grid square's inclusion (upper left corner, lower right corner, the center, etc.), but if a gate boundary includes that location then all events in the grid are included in the gate, regardless of whether their actual coordinate location is just outside the gate. Normally this wouldn't be a huge problem, but for a case like this it is.

Another problem is that the FlowJo plot view ranges are not independent of the transform parameters, i.e. you can't "zoom" out or in. Meaning it's not easy to zoom out for a case like this (at least not that I'm aware) to make sure your boundary includes those "floored" events if you intend to include them.

There are several issues in this repo where I discuss the binning issue (#113 for example). A search for "binning" will probably find more.

I think that does explain this. One more question: is this data using conventional compensation or spectral?

[hbhargava7]

Makes sense.

I notice that (on quick inspection) the gate def in the wsp file appears to be able to go far negative

                         <gating:dimension gating:min="-1748.7027644514837"  gating:max="363724.64110574685" >
                           <data-type:fcs-dimension data-type:name="Comp-Alexa Fluor 700-A" />
                         </gating:dimension>

Do you know why it gets reset to zero by the time it's a Dimension object? The implementation is a bit opaque to me due to the transform/rescaling stuff.

This is conventional compensation.

[whitews]

Sorry, I didn't see your last post. Gate boundaries are inclusive so events falling exactly on the boundary are included in FlowKit (per the GatingML specification).

A couple more things. Can you post a pip freeze output so I can see the FlowKit dependency versions you have? And would it be possible to share the .wsp file (privately if needed)? I do want to make sure FlowKit is not at fault here.

[hbhargava7]

If gate bounds are inclusive then why do you think they are seemingly getting gated out?

I can get you the wsp file but will strip it down to a minimal example first (for your convenience and our confidentiality)

Rather a long pip freeze, sorry I have some other stuff in this env

adjustText==1.3.0
anytree==2.13.0
appnope==0.1.4
asttokens==3.0.0
bokeh==3.8.0
comm==0.2.3
contourpy==1.3.3
cycler==0.12.1
debugpy==1.8.16
decorator==5.2.1
et_xmlfile==2.0.0
executing==2.2.0
FlowIO==1.3.0
FlowKit==1.2.3
FlowUtils==1.1.0
fonttools==4.59.1
iniconfig==2.1.0
ipykernel==6.30.1
ipython==9.4.0
ipython_pygments_lexers==1.1.1
jedi==0.19.2
Jinja2==3.1.6
jupyter_client==8.6.3
jupyter_core==5.8.1
kiwisolver==1.4.9
lxml==6.0.0
MarkupSafe==3.0.2
matplotlib==3.7.3
matplotlib-inline==0.1.7
narwhals==2.10.2
nest-asyncio==1.6.0
networkx==3.5
numpy==1.26.4
openpyxl==3.1.5
packaging==25.0
pandas==2.3.3
parso==0.8.4
pexpect==4.9.0
pillow==11.3.0
platformdirs==4.3.8
pluggy==1.6.0
prompt_toolkit==3.0.51
psutil==7.0.0
ptyprocess==0.7.0
pure_eval==0.2.3
Pygments==2.19.2
pyparsing==3.2.3
pytest==8.4.1
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==6.0.2
pyzmq==27.0.2
scipy==1.14.1
seaborn==0.11.2
six==1.17.0
stack-data==0.6.3
tornado==6.5.2
traitlets==5.14.3
typing_extensions==4.14.1
tzdata==2025.2
wcwidth==0.2.13
xyzservices==2025.4.0

[whitews]

Thanks, all those deps look ok. You can send the file to my work email: scott.white@duke.edu

That min value of -1748 is why I want to look at the file, to trace through the code to see it's exact numerical value after parsing.