Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import torchgeo.datasets is slow #2589

Open
calebrob6 opened this issue Feb 18, 2025 · 2 comments
Open

import torchgeo.datasets is slow #2589

calebrob6 opened this issue Feb 18, 2025 · 2 comments
Labels
datasets Geospatial or benchmark datasets

Comments

@calebrob6
Copy link
Member

Description

python -X importtime -c "import torchgeo.datasets" takes 29 seconds for me. We may want to switch to a dynamic/lazy import pattern. Happy to implement this if we can agree on a pattern!

Steps to reproduce

see above

Version

latest

@adamjstewart
Copy link
Collaborator

> time python3 -c 'import torchgeo.datasets'

________________________________________________________
Executed in    1.76 secs    fish           external
   usr time    1.47 secs    0.10 millis    1.47 secs
   sys time    0.28 secs    2.10 millis    0.28 secs

Not great, but much faster than 29 sec. Roughly 3/4 of that time is taken up by import torch. I don't think lazy imports are possible in that case, since we need torch.utils.data.Dataset at a bare minimum.

@adamjstewart adamjstewart added the datasets Geospatial or benchmark datasets label Feb 19, 2025
@calebrob6
Copy link
Member Author

If I do that, then I get numbers ~7 seconds

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Geospatial or benchmark datasets
Projects
None yet
Development

No branches or pull requests

2 participants