Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unexpected floats when reading LI L2 LFL data #2854

Open
gerritholl opened this issue Jul 10, 2024 · 3 comments · May be fixed by #2998
Open

Unexpected floats when reading LI L2 LFL data #2854

gerritholl opened this issue Jul 10, 2024 · 3 comments · May be fixed by #2998
Labels
bug component:readers

Comments

@gerritholl
Copy link
Member

Describe the bug

When reading LI L2 LFL data, several datasets are returned as floats when they should be integers.

To Reproduce

from satpy import Scene
from glob import glob
from satpy.utils import debug_on; debug_on()

fn = "/media/nas/x23352/MTG/LI/L2/2024/07/01/10/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LFL--FD--CHK-BODY---NC4E_C_EUMT_20240701105944_L2PF_OPE_20240701105923_20240701105933_N__T_0066_0057.nc"
sc = Scene(filenames=[fn], reader="li_l2_nc")
sc.load(["flash_id"])
print(sc["flash_id"].dtype)

Expected behavior

I expect an integer dtype.

Actual results

I get float64.

[DEBUG: 2024-07-10 16:42:30 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2_nc.yaml',)
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.yaml_reader] Assigning to li_l2_nc: ['/media/nas/x23352/MTG/LI/L2/2024/07/01/10/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LFL--FD--CHK-BODY---NC4E_C_EUMT_20240701105944_L2PF_OPE_20240701105923_20240701105933_N__T_0066_0057.nc']
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_base_nc] Product type is: 2-LFL
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_base_nc] Adding 12 datasets for 2-LFL input product.
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_l2_nc] The current product 2-LFL is not an accumulated product so it will not be regridded.
[DEBUG: 2024-07-10 16:42:30 : satpy.composites.config_loader] Looking for composites config file li.yaml
[DEBUG: 2024-07-10 16:42:30 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2024-07-10 16:42:30 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/pyorbital/etc
[DEBUG: 2024-07-10 16:42:31 : satpy.readers.yaml_reader] No coordinates found for DataID(name='longitude', modifiers=())
[DEBUG: 2024-07-10 16:42:31 : satpy.readers.yaml_reader] No coordinates found for DataID(name='latitude', modifiers=())
float64

Environment Info:

  • OS: [e.g. OSX, Windows, Linux]
  • Satpy Version: main (v0.49.0-156-g3a742c297)

Additional context

The same for other integer variables in the dataset.

Sometimes this happens when a fill value is defined in the variable attributes, but that is not the case here:

	uint flash_id(flashes) ;
		flash_id:_Unsigned = "true" ;
		flash_id:long_name = "LI L2 Flash IDs" ;
@mraspaud
Copy link
Member

@ameraner ^

@ameraner ameraner mentioned this issue Oct 4, 2024
Open
@ClementLaplace
Copy link
Contributor

ClementLaplace commented Nov 21, 2024 •
edited
Loading

@ameraner @gerritholl this issue happens in this method
this line bellow makes the transformation

data_array = (data_array * scale_factor) + add_offset

whith the following type

  • data_array : uint32
  • scale_factor : float32

Or with python the multiplication of a uint32 with a float32 get a float64

I suggest that we add this modification into the apply_use_rescaling method

if data_array.dtype == "uint32":
    data_array = data_array.astype(type(scale_factor))

@ameraner
Copy link
Member

Is this also the case for flash_id specifically? I am surprised it has a scale factor.. what is the value of it?

@ClementLaplace ClementLaplace linked a pull request Nov 22, 2024 that will close this issue
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug component:readers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Correct Unexpected floats when reading LI L2 LFL ClementLaplace/satpy
4 participants
@mraspaud @gerritholl @ameraner @ClementLaplace