Skip to content

Regression tests: cache round-trip and gcsfs mock helper - #97

Merged
maritsandstad merged 2 commits into
basefrom
tests/cmip6-data-getter-regression-tests
Aug 3, 2026
Merged

Regression tests: cache round-trip and gcsfs mock helper#97
maritsandstad merged 2 commits into
basefrom
tests/cmip6-data-getter-regression-tests

Conversation

@benmsanderson

Copy link
Copy Markdown
Owner

Summary

Adds three focused regression tests targeting the class of bug where the outer cache validator's expected filename and the inner save/load filename diverge — silently triggering redundant network fetches on every call. That was the shape of the bug fixed in the sibling cache-validator PR; this hardens the boundary so a repeat lands in CI, not production.

No src/ changes. Tests only.

What's in

1. Pattern-scaling cache-path round-trip (tests/unit/test_cache_handler.py)

For each of tas / pr, asserts that
CacheHandler.get_pattern_scaling_cache_path(model, variable=v) is byte-equal to the location MeteorPatternScaling.save_model writes to when instantiated with the corresponding per-variable name (cmip6-{model}-aer-{v}). This is the smoking-gun test for the class of bug: any drift between the two sides of the write/validate/read boundary now fails at CI time.

2. Composite training data contiguity (tests/unit/test_cmip6_meteor_data_getter.py)

Uses the existing light_mock_cache fixture to verify that make_meteor_training_data_composite(["historical", "ssp370"], "CanESM5") produces 251 strictly contiguous years with no gaps, no duplicates, and no NaN gaps in either variable. Existing tests check the size but not the join — an off-by-one at the boundary would pass those but corrupt training silently.

3. gcs_zarr_mock fixture + fetch-and-cache round-trip

Introduces a small fixture that patches the two network seams the data-getter uses (gcsfs.GCSFileSystem and xarray.open_zarr) and yields a synthetic monthly CMIP6-shape dataset. Tests can now drive the fetch path without a network.

Uses it to assert that a cache-miss fetch writes an .nc file at the exact path _generate_cmip6_cache_key predicts, and a second call is served from the cache without re-consulting gcsfs. Any divergence between the write filename and the validate/read filename shows up here as either a missing file or a redundant fetch.

Why this matters for CI resilience

The existing test suite avoids the network by leaning on a pre-populated disk cache (~40 real netCDFs in tests/test-data/light_mock_cache/). That's fast but only exercises the cache-hit path. The gcs_zarr_mock fixture opens up the fetch path to unit-testing, which is where cache-boundary bugs like the one in the sibling PR live. Follow-up work could extend it to test the catalog HTTPS fallback and NaN-zstore paths without touching the network.

Verified

  • All 35 tests in test_cache_handler.py + test_cmip6_meteor_data_getter.py pass.
  • black, ruff, pylint 10.00/10 all clean.
  • No src/ changes; zero behavior risk.

Test plan

  • pytest tests/unit/test_cache_handler.py::test_pattern_scaling_cache_path_roundtrips_meteor_pattern_scaling_save_name
  • pytest tests/unit/test_cmip6_meteor_data_getter.py::test_make_meteor_training_data_composite_stitches_historical_and_ssp_contiguously
  • pytest tests/unit/test_cmip6_meteor_data_getter.py::test_get_single_var_mod_data_monthly_writes_cache_at_expected_path
  • Full sweep: pytest tests/unit/test_cache_handler.py tests/unit/test_cmip6_meteor_data_getter.py

🤖 Generated with Claude Code

Adds three regression tests targeting the class of bug where the outer
cache validator's expected filename and the inner save/load filename
diverge. Symptom of the class: outer view reports "cache missing" for a
file that the inner loader then reads successfully by name, silently
triggering redundant fetches on every call.

1. test_pattern_scaling_cache_path_roundtrips_meteor_pattern_scaling_save_name
   (test_cache_handler.py)
   For each of tas / pr, asserts that
   CacheHandler.get_pattern_scaling_cache_path(model, variable=v) is
   byte-equal to the location MeteorPatternScaling.save_model writes to
   when instantiated with the per-variable name (cmip6-{model}-aer-{v}).

2. test_make_meteor_training_data_composite_stitches_historical_and_ssp_contiguously
   (test_cmip6_meteor_data_getter.py)
   Uses the existing light_mock_cache fixture to verify that a
   historical + ssp370 composite for CanESM5 produces 251 strictly
   contiguous years with no gaps, no duplicates, and no NaN gaps in
   either tas or pr. Existing tests check the size but not the join.

3. test_get_single_var_mod_data_monthly_writes_cache_at_expected_path
   (test_cmip6_meteor_data_getter.py)
   Introduces gcs_zarr_mock (fixture) which patches the two network
   seams the data-getter uses: gcsfs.GCSFileSystem and xarray.open_zarr.
   The fixture yields a small synthetic monthly CMIP6-shape dataset so
   tests can drive the fetch-and-cache path without a network. The test
   then asserts a fresh-cache fetch writes .nc at the exact key
   _generate_cmip6_cache_key predicts, and a second call is served from
   cache without re-consulting gcsfs.

All 35 tests in the two affected files pass. No src/ changes; this PR
is purely tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@maritsandstad maritsandstad left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, I'll fix linting and merge in

@maritsandstad
maritsandstad merged commit 46f2d9f into base Aug 3, 2026
16 checks passed
@maritsandstad
maritsandstad deleted the tests/cmip6-data-getter-regression-tests branch August 3, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants