From c77b1b28efc5c47ccac9702263c20ed31ee26b73 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 17 Sep 2025 18:59:43 +0000 Subject: [PATCH 1/2] Create comprehensive GitHub Copilot instructions with validated commands Co-authored-by: xylar <4179064+xylar@users.noreply.github.com> --- .github/copilot-instructions.md | 151 ++++++++++++++++++ .../make_mpas_to_antarctic_stereo_mapping.py | 4 +- examples/make_mpas_to_lat_lon_mapping.py | 1 - ...mpas_vertex_to_antarctic_stereo_mapping.py | 9 +- examples/remap_stereographic.py | 51 ++++-- 5 files changed, 193 insertions(+), 23 deletions(-) create mode 100644 .github/copilot-instructions.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..9addaad7 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,151 @@ +# pyremap + +pyremap is a Python library for remapping climate and earth system model data between different grids and projections. It provides tools for creating mapping files and performing remapping operations using ESMF (Earth System Modeling Framework) and ncremap. + +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. + +## Working Effectively + +### Bootstrap, build, and test the repository: +- Setup conda environment: + ```bash + conda config --add channels conda-forge + conda config --set channel_priority strict + conda create -y -n pyremap --file dev-spec.txt + ``` + - NEVER CANCEL: Environment creation takes 15-20 minutes. Set timeout to 30+ minutes. +- Activate environment and install package: + ```bash + source /usr/share/miniconda/etc/profile.d/conda.sh # if conda init wasn't run + conda activate pyremap + python -m pip install --no-deps --no-build-isolation -e . + ``` + - Takes ~30 seconds to complete. +- Run tests: + ```bash + pytest --pyargs tests + ``` + - NEVER CANCEL: Test suite takes ~15 seconds for 18 tests. Set timeout to 5+ minutes. +- Run tests with verbose output: + ```bash + pytest --pyargs tests -v + ``` + +### Build documentation: +- Build docs: + ```bash + cd docs + DOCS_VERSION=main make versioned-html + ``` + - NEVER CANCEL: Documentation build takes ~4 minutes. Set timeout to 10+ minutes. +- Preview docs locally: + ```bash + cd _build/html + python -m http.server 8000 + # Open http://0.0.0.0:8000/main/ in browser + ``` + +### Linting and code quality: +- Run ruff linting check: + ```bash + ruff check . + ``` + - Takes ~0.01 seconds. +- Auto-fix linting issues: + ```bash + ruff check --fix . + ``` +- Format code: + ```bash + ruff format . + ``` + - Takes ~0.01 seconds. +- Run type checking: + ```bash + mypy --config=pyproject.toml pyremap + ``` + +## Validation + +- ALWAYS run through at least one complete end-to-end scenario after making changes. +- Test basic functionality by running: + ```python + from pyremap import Remapper, get_lat_lon_descriptor + descriptor = get_lat_lon_descriptor(dlon=5.0, dlat=5.0) + remapper = Remapper(ntasks=1, method='bilinear') + remapper.src_descriptor = descriptor + remapper.dst_descriptor = descriptor + print("Basic functionality working") + ``` +- ALWAYS run `ruff check --fix .` and `ruff format .` before you are done or the CI (.github/workflows/build_workflow.yml) will fail. +- ALWAYS run the test suite with `pytest --pyargs tests` after making changes. + +## Common tasks + +The following are outputs from frequently run commands. Reference them instead of viewing, searching, or running bash commands to save time. + +### Repository root structure +``` +.github/ # GitHub workflows and configuration +ci/ # Conda build recipes +dev-spec.txt # Development dependencies for conda +docs/ # Sphinx documentation +examples/ # Example scripts +pyremap/ # Main Python package +tests/ # Test suite +pyproject.toml # Project configuration +README.md # Project readme +``` + +### Key package modules +``` +pyremap/ +├── __init__.py # Main package imports +├── descriptor/ # Grid descriptors (MPAS, lat-lon, projections) +├── remapper/ # Core remapping functionality +├── polar.py # Polar projection utilities +├── utility.py # Helper functions +└── version.py # Version information +``` + +### Available external tools +- `ESMF_RegridWeightGen`: Available for creating mapping files with ESMF +- `ncremap`: Available for remapping NetCDF files +- `mbtempest`: NOT available (MOAB tool for mapping) + +### Common pyremap workflows +1. **Create mapping file**: Use `Remapper.build_map()` to generate mapping weights +2. **Remap with ncremap**: Use `Remapper.ncremap()` to remap NetCDF files +3. **Remap with numpy**: Use `Remapper.remap_numpy()` for in-memory remapping +4. **Grid descriptors**: Use classes like `LatLonGridDescriptor`, `MpasCellMeshDescriptor`, etc. + +### Dependencies and requirements +- Python >=3.9 +- Core dependencies: dask, netcdf4, numpy, pyproj, scipy, xarray >=0.10.0 +- External tools: ESMF (included), NCO (included for ncremap) +- Development tools: pytest, ruff, mypy, sphinx + +### CI/CD information +- Tests run on Python 3.9-3.13 +- Build timeout: 20 minutes +- Pre-commit hooks: ruff (linting/formatting), mypy (type checking) +- Documentation is auto-deployed from main branch + +### Timing expectations +- **Environment setup**: 15-20 minutes (conda create) +- **Package installation**: ~30 seconds +- **Test suite**: ~15 seconds (18 tests) +- **Documentation build**: ~4 minutes +- **Linting/formatting**: <1 second +- **Basic import**: <1 second + +### Manual validation scenarios +- Create a basic lat-lon grid descriptor: `get_lat_lon_descriptor(dlon=5.0, dlat=5.0)` +- Create a remapper instance: `Remapper(ntasks=1, method='bilinear')` +- Set source and destination grids +- Verify grid properties: `descriptor.mesh_name`, `descriptor.dims`, `descriptor.dim_sizes` + +### Known limitations +- MOAB tools (mbtempest) are not available in the conda environment +- Network-dependent operations (intersphinx, pre-commit remote hooks) may fail in restricted environments +- Some examples require external MPAS grid files that are not included in the repository \ No newline at end of file diff --git a/examples/make_mpas_to_antarctic_stereo_mapping.py b/examples/make_mpas_to_antarctic_stereo_mapping.py index 4fdc0fe1..1ce4eb5e 100755 --- a/examples/make_mpas_to_antarctic_stereo_mapping.py +++ b/examples/make_mpas_to_antarctic_stereo_mapping.py @@ -23,7 +23,6 @@ from pyremap import Remapper, get_polar_descriptor - # replace with the MPAS mesh name src_mesh_name = 'oQU240' @@ -40,7 +39,8 @@ # modify the size and resolution of the Antarctic grid as desired remapper.dst_descriptor = get_polar_descriptor( - lx=6000., ly=5000., dx=10., dy=10., projection='antarctic') + lx=6000.0, ly=5000.0, dx=10.0, dy=10.0, projection='antarctic' +) dst_grid_name = remapper.dst_descriptor.mesh_name diff --git a/examples/make_mpas_to_lat_lon_mapping.py b/examples/make_mpas_to_lat_lon_mapping.py index 59e1dece..77919dca 100755 --- a/examples/make_mpas_to_lat_lon_mapping.py +++ b/examples/make_mpas_to_lat_lon_mapping.py @@ -23,7 +23,6 @@ from pyremap import Remapper, get_lat_lon_descriptor - # replace with the MPAS mesh name src_mesh_name = 'oQU240' diff --git a/examples/make_mpas_vertex_to_antarctic_stereo_mapping.py b/examples/make_mpas_vertex_to_antarctic_stereo_mapping.py index 964aa3d1..717c746c 100755 --- a/examples/make_mpas_vertex_to_antarctic_stereo_mapping.py +++ b/examples/make_mpas_vertex_to_antarctic_stereo_mapping.py @@ -24,7 +24,6 @@ from pyremap import Remapper, get_polar_descriptor - # replace with the MPAS mesh name src_mesh_name = 'oQU240_vertex' @@ -37,12 +36,14 @@ # conservative remapping remapper = Remapper(ntasks=4, method='conserve') -remapper.src_from_mpas(filename=src_mesh_filename, mesh_name=src_mesh_name, - mesh_type='vertex') +remapper.src_from_mpas( + filename=src_mesh_filename, mesh_name=src_mesh_name, mesh_type='vertex' +) # modify the size and resolution of the Antarctic grid as desired remapper.dst_descriptor = get_polar_descriptor( - lx=6000., ly=5000., dx=10., dy=10., projection='antarctic') + lx=6000.0, ly=5000.0, dx=10.0, dy=10.0, projection='antarctic' +) dst_grid_name = remapper.dst_descriptor.mesh_name diff --git a/examples/remap_stereographic.py b/examples/remap_stereographic.py index 7de11bed..9f8bfb55 100755 --- a/examples/remap_stereographic.py +++ b/examples/remap_stereographic.py @@ -14,23 +14,40 @@ import numpy as np import xarray as xr -from pyremap import Remapper, ProjectionGridDescriptor +from pyremap import ProjectionGridDescriptor, Remapper from pyremap.polar import get_antarctic_stereographic_projection - parser = argparse.ArgumentParser( - description=__doc__, formatter_class=argparse.RawTextHelpFormatter) -parser.add_argument('-i', dest='in_filename', required=True, type=str, - help='Input file name') -parser.add_argument('-o', dest='out_filename', required=True, type=str, - help='Output file name') -parser.add_argument('-r', dest='resolution', required=True, type=float, - help='Output resolution') -parser.add_argument('-m', dest='method', required=False, default='bilinear', - help='Method: {"bilinear", "neareststod", "conserve"}') -parser.add_argument('-t', dest='mpi_tasks', required=False, type=int, - default=1, - help='Number of MPI tasks (default = 1)') + description=__doc__, formatter_class=argparse.RawTextHelpFormatter +) +parser.add_argument( + '-i', dest='in_filename', required=True, type=str, help='Input file name' +) +parser.add_argument( + '-o', dest='out_filename', required=True, type=str, help='Output file name' +) +parser.add_argument( + '-r', + dest='resolution', + required=True, + type=float, + help='Output resolution', +) +parser.add_argument( + '-m', + dest='method', + required=False, + default='bilinear', + help='Method: {"bilinear", "neareststod", "conserve"}', +) +parser.add_argument( + '-t', + dest='mpi_tasks', + required=False, + type=int, + default=1, + help='Number of MPI tasks (default = 1)', +) args = parser.parse_args() @@ -51,7 +68,8 @@ remapper = Remapper(ntasks=args.mpi_tasks, method=args.method) remapper.src_descriptor = ProjectionGridDescriptor.create( - projection, x, y, src_mesh_name) + projection, x, y, src_mesh_name +) out_res = args.resolution * 1e3 @@ -64,7 +82,8 @@ dst_mesh_name = f'{lx}x{ly}km_{args.resolution}km_Antarctic_stereo' remapper.dst_descriptor = ProjectionGridDescriptor.create( - projection, x_out, y_out, dst_mesh_name) + projection, x_out, y_out, dst_mesh_name +) remapper.build_map() From c263e348f2b353e0a03f0774e7d503270f6e38dd Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 17 Sep 2025 19:06:02 +0000 Subject: [PATCH 2/2] Fix conda environment setup timing from 15-20min to 1-2min Co-authored-by: xylar <4179064+xylar@users.noreply.github.com> --- .github/copilot-instructions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 9addaad7..ac353827 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -13,7 +13,7 @@ Always reference these instructions first and fallback to search or bash command conda config --set channel_priority strict conda create -y -n pyremap --file dev-spec.txt ``` - - NEVER CANCEL: Environment creation takes 15-20 minutes. Set timeout to 30+ minutes. + - NEVER CANCEL: Environment creation takes ~1-2 minutes. Set timeout to 5+ minutes. - Activate environment and install package: ```bash source /usr/share/miniconda/etc/profile.d/conda.sh # if conda init wasn't run @@ -132,7 +132,7 @@ pyremap/ - Documentation is auto-deployed from main branch ### Timing expectations -- **Environment setup**: 15-20 minutes (conda create) +- **Environment setup**: ~1-2 minutes (conda create) - **Package installation**: ~30 seconds - **Test suite**: ~15 seconds (18 tests) - **Documentation build**: ~4 minutes