Skip to content

Conversation

@nicHoch
Copy link
Contributor

@nicHoch nicHoch commented Apr 10, 2025

add a internal ephemeris data cache. This allows to bypass hidden internal FIDO searches and downloads of the same data multiple times

All coordinate transform and imaging methods might internally search for correct ephemeris data for higher precession.
To get the ephemeris data a FIDO search (defaults to our stix data server) is uses and data gets downloaded.

The ephemeris data search utilises a global cache in order to avoid unnecessary network traffic or file loading. ANC ephemeris fits files are organised as daily files. The entire day is put to cache if touched once.

A user can prefetch or pinpoint specific ephemeris data files with the `load_ephemeris_fits_to_cache(filename)' method.

    from stixpy.coordinates.transforms import load_ephemeris_fits_to_cache

    start_time = Time("2023-01-01T12:00:00")
    end_time = Time("2023-01-01T12:30:00")
    query = Fido.search(
        a.Time(start_time, end_time),
        a.Instrument.stix,
        a.Level.anc,
        a.stix.DataType.asp,
        a.stix.DataProduct.asp_ephemeris,
    )

    aux_files = Fido.fetch(query["stix"])
    for file in aux_files:
        load_ephemeris_fits_to_cache(file)

…search and download of the same data multiple times
@nicHoch nicHoch self-assigned this Apr 10, 2025
@codecov
Copy link

codecov bot commented Apr 10, 2025

Codecov Report

❌ Patch coverage is 89.04110% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.42%. Comparing base (f6b4a88) to head (6258511).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
stixpy/product/sources/quicklook.py 0.00% 5 Missing ⚠️
stixpy/product/sources/anc.py 87.09% 4 Missing ⚠️
stixpy/_dev/scm_version.py 50.00% 3 Missing ⚠️
stixpy/coordinates/transforms.py 92.85% 3 Missing ⚠️
stixpy/utils/table_lru.py 94.73% 3 Missing ⚠️
stixpy/calibration/visibility.py 83.33% 2 Missing ⚠️
stixpy/product/sources/housekeeping.py 0.00% 2 Missing ⚠️
stixpy/calibration/transmission.py 0.00% 1 Missing ⚠️
stixpy/timeseries/quicklook.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #165      +/-   ##
==========================================
+ Coverage   75.50%   81.42%   +5.91%     
==========================================
  Files          32       37       +5     
  Lines        2111     2498     +387     
==========================================
+ Hits         1594     2034     +440     
+ Misses        517      464      -53     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samaloney samaloney force-pushed the anc_file_skip_auto_fido_search branch from 7bc0a76 to 1a8cd28 Compare August 6, 2025 16:46
Copy link
Member

@samaloney samaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this is very different from what I was thinking but that not a bad thing, any idea on performance? As the reason I added the cache in the first place was because this code is call a lot during plotting operations.

It would be nice to be disable the cache maybe add a module level variable. Needs some narrative documentation could be added to top of the rst file and also needs a corresponding change log.

In the future if we have different version of the anc-ephem files and I wanted to compare the results from V0X to V0Y is this in the table or does the user have to remember which one the loaded last so will be in the cache? It would probably be useful to know which file the data came from anyway for debugging purposes anyway

@samaloney
Copy link
Member

@nicHoch I force pushed, sorry, so don't forget to pull the changes back down to your local branch.

@samaloney
Copy link
Member

pre-commit.ci autofix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants