Skip to content

Conversation

@Antsalacia
Copy link
Collaborator

Added a compression option in set_query_parameters and put new key in the output_metadata dictionary.
Method get_raster_compression added with a try and except.

@wehrad
Copy link
Owner

wehrad commented Nov 28, 2024

Thanks for working on this @Antsalacia! You create a get_raster_compression() method but never use it.

I guess you'd want this:

https://github.com/AdrienWehrle/earthspy/blob/71b3cc5afd7690a417420e1b048b99ced220c663/earthspy/earthspy.py#L178

to rather be something like:

self.get_raster_compression()

with self.compression being set inside the method, just like e.g. :

https://github.com/AdrienWehrle/earthspy/blob/71b3cc5afd7690a417420e1b048b99ced220c663/earthspy/earthspy.py#L204-L219

@Antsalacia Antsalacia marked this pull request as ready for review November 29, 2024 10:35
@Antsalacia
Copy link
Collaborator Author

Solved #84

@wehrad
Copy link
Owner

wehrad commented Nov 29, 2024

For now this code isn't working, but it's not visible since we can't run earthspy properly in tests at the moment (I'm working on getting credentials for testing purposes).

That's because in get_raster_compression() you're calling a compress_mode which is not passed as an argument to the function. So you need to pass compression_mode in the function just like evaluation_script is passed in self.get_evaluation_script(evaluation_script) right below.

Actually compress_mode doesn't exist anywhere else. You have a compression variable in set_query_parameters but compress_mode doesn't exist anywhere else than in the function.

Remember to run your modifications before pushing your code! If it doesn't work, please make a note that it will be fixed later! :)

@wehrad
Copy link
Owner

wehrad commented Nov 29, 2024

Also, could you:

  • rename compression for raster_compression
  • add documentation for the new raster_compression argument in the docstring of set_query_parameters
  • add a few tests for the new raster_compression in test_earthspy.py

@wehrad wehrad self-requested a review November 29, 2024 11:02
@Antsalacia
Copy link
Collaborator Author

What kind of test should I add in test_earthspy for raster_compression ?

@wehrad wehrad added the enhancement New feature or request label Nov 29, 2024
@wehrad
Copy link
Owner

wehrad commented Nov 30, 2024

Have you tested if the raster saving works fine with "compress" to None in the metadata? You could try on the earthspy example in the README.

@Antsalacia
Copy link
Collaborator Author

The raster saving works fine when no argument is pass to raster_compress and thus equal to None

@wehrad
Copy link
Owner

wehrad commented Dec 2, 2024

The raster saving works fine when no argument is pass to raster_compress and thus equal to None

Cool thanks!

Then what remains is to resolve the discussion with CodeQL further up here and then you'd need to run the pre-commit hooks and pytest.

See CONTRIBUTING.md for details. :)

@wehrad
Copy link
Owner

wehrad commented Dec 2, 2024

I just added a commit to address CodeQL alert that raster_compression == None should be raster_compression is None.

I see you fixed pre-commit issues, do you still have any or does everything pass? Is pytest also passing all tests?

@wehrad
Copy link
Owner

wehrad commented Dec 2, 2024

Don't hesitate to git add all the files modified by black once you've run pre-commit. As you can see here black would like to format the code but you haven't pushed the results.

One would usually run pre-commit run --al-files twice, first to format, then to make sure formatting fixed all issues.

@wehrad
Copy link
Owner

wehrad commented Dec 2, 2024

closes #84

@wehrad
Copy link
Owner

wehrad commented Dec 2, 2024

Now that I think about it, we could only update the metadata dictionary to add the compression when raster_compression is not none, so that it doesn't unnecessarily write compression to None.

@wehrad
Copy link
Owner

wehrad commented Dec 12, 2024

I just added more comments and fixed a couple more things.

So I think the only thing missing here is to get test_get_raster_compression ton run on t3 and t4!

@Antsalacia
Copy link
Collaborator Author

How can I check raster_compression when I try this it doesn't work:
comp_def = t4.get_raster_compression()
error:
AttributeError: 'function' object has no attribute 'get_raster_compression'

@wehrad
Copy link
Owner

wehrad commented Dec 16, 2024

Maybe earthspy hasn't been reloaded. Can you try pip install -e . and then again?

@Antsalacia
Copy link
Collaborator Author

Still the same after pip

@wehrad
Copy link
Owner

wehrad commented Dec 16, 2024

Can you share where you're running this t4? Can you try on a simple example too where you just setup a job and then try access the function?

@Antsalacia
Copy link
Collaborator Author

I am running it in the test_get_raster_compression and t4 is in conftest. It works with the job.

@Antsalacia
Copy link
Collaborator Author

All good now was just testing the wrong way thanks for helping.

@wehrad
Copy link
Owner

wehrad commented Dec 16, 2024

Nice to hear! Now please modify the tests to compliy with github-advanced-security

# assert all(isinstance(item, str) for item in self.t3.output_filenames_renamed)
# # check that one output per split box was created
# assert len(self.t3.outputs) == len(self.t3.split_boxes)
# check raster compression sith mode specified
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a typo left :) "sith"

@wehrad wehrad merged commit 15d3c93 into wehrad:main Dec 17, 2024
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants