Skip to content

Add river-network preprocessing for the unified MPAS mesh#556

Draft
xylar wants to merge 6 commits intoE3SM-Project:mainfrom
xylar:add-prepare-river-network
Draft

Add river-network preprocessing for the unified MPAS mesh#556
xylar wants to merge 6 commits intoE3SM-Project:mainfrom
xylar:add-prepare-river-network

Conversation

@xylar
Copy link
Copy Markdown
Collaborator

@xylar xylar commented Apr 22, 2026

Summary

This PR adds polaris/tasks/mesh/spherical/unified/river as the river-network
preprocessing path for the unified spherical mesh workflow, together with unit
tests covering both source-level simplification and lat-lon target-grid
products.

The implementation introduces shared source-level and lat-lon river workflows,
uses HydroRIVERS as the public source dataset, and adds visualization and
diagnostic outputs to make the retained network and outlet matching easier to
inspect.

What This PR Adds

  • a shared source-level river workflow that downloads HydroRIVERS, unpacks the archive, converts the shapefile to GeoJSON, and writes:
    • source_river_network.geojson
    • simplified_river_network.geojson
    • retained_outlets.geojson
  • a source-level simplification algorithm that:
    • filters segments by drainage area
    • validates that retained NEXT_DOWN links are acyclic
    • suppresses nearby duplicate ocean outlets
    • preserves inland sinks
    • retains a basin-aware simplified network with main stems and major tributaries
  • a shared lat-lon river workflow that rasterizes the simplified network onto
    the supported unified target grids and writes:
    • river_network.nc
    • river_outlets.geojson
  • target-grid outputs that separate river semantics into explicit masks:
    • river_channel_mask
    • river_outlet_mask
    • river_ocean_outlet_mask
    • river_inland_sink_mask
  • outlet reconciliation against shared coastline products, including snapped outlet diagnostics and matched/unmatched ocean-outlet metadata
  • shared-step factories and standalone task wrappers for:
    • the source-level workflow
    • each supported lat-lon unified target-grid resolution
  • a diagnostic visualization step for inspecting the retained network and snapped outlets
  • package exports and river_network.cfg settings for source download, simplification, outlet matching, and rasterization

Testing

This PR adds tests/mesh/spherical/unified/test_river.py with coverage for:

  • source-level simplification retaining major outlets and tributaries
  • traversal of deep main stems
  • rejection of cyclic NEXT_DOWN graphs
  • branch-order behavior at confluences
  • the lat-lon dataset contract and snapped-outlet metadata
  • inland-sink snapping and unmatched ocean-outlet handling
  • HydroRIVERS archive unpacking and shapefile-to-GeoJSON conversion

Notes

The river simplification is implemented as a focused Polaris-native
reimplementation rather than a direct port of the standalone workflow internals.
The current behavior is designed to preserve basin connectivity, dominant
outlets, and major tributaries while keeping the workflow compact and testable.

Checklist

  • User's Guide has been updated
  • Developer's Guide has been updated
  • API documentation in the Developer's Guide (api.md) has any new or modified class, method and/or functions listed
  • Documentation has been built locally and changes look as expected
  • Testing comment in the PR documents testing used to verify the changes
  • New tests have been added to a test suite

@xylar xylar self-assigned this Apr 22, 2026
@xylar xylar added mesh Related to the mesh component unified-mesh enhancement New feature or request labels Apr 22, 2026
@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented Apr 22, 2026

This is based off of #545

@xylar xylar force-pushed the add-prepare-river-network branch from 9b31622 to 970d488 Compare April 23, 2026 09:16
@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented Apr 24, 2026

Testing

I ran the 3 unified mesh tasks on Frontier:

  mesh/spherical/unified/river/ocn_240km_lnd_240km_riv_240km/lat_lon/task
  mesh/spherical/unified/river/ocn_30km_lnd_10km_riv_10km/lat_lon/task
  mesh/spherical/unified/river/ocn_rrs_6to18km_lnd_12km_riv_6km/lat_lon/task

Here are the resulting river networks and outlets.

ocn_240km_lnd_240km_riv_240km:
river_network_overview

ocn_30km_lnd_10km_riv_10km:
river_network_overview

ocn_rrs_6to18km_lnd_12km_riv_6km:
river_network_overview

No doubt, I didn't set the parameters the way that they need to be, and we'll need to iterate but the pieces would seem to be in place.

@xylar xylar force-pushed the add-prepare-river-network branch 7 times, most recently from d721849 to 324c1dd Compare April 28, 2026 16:44
@xylar xylar force-pushed the add-prepare-river-network branch 5 times, most recently from df5c589 to 83badd1 Compare May 8, 2026 12:07
xylar added 5 commits May 8, 2026 14:29
This workflow extracts a signed distance function and ocean/land mask from a combined topography dataset on a lat-lon grid.

Supported resolutions are 1/4, 1/8, 1/16 and 1/32 degree.

It supports three different approaches to the land-ocean interface in Antarctica:
1. coastline at the calving front (main supported case for now)
2. coastline at the grounding (future meshes with ice-shelf cavities) ; and
3. coastline at zero bedrock elevation (potential future meshes with wetting-and-drying).

The ocean region is flood-filled from the north pole to ensure that it is contiguous.

It uses critical passages and blockages to ensure that the floodfill includes or excludes the same regions as we want in the final ocean mesh.
Add the unified spherical river workflow used to prepare HydroRIVERS data for mesh generation and analysis.

- add unified-mesh river configuration and shared defaults under polaris.mesh.spherical.unified
- add source, lat-lon, task, step, and visualization plumbing for preparing and inspecting river products
- integrate coastline-aware rasterization and shared HydroRIVERS archiving for the retained network
- add base-mesh river conditioning so downstream mesh workflows can consume clipped and simplified river products
Add and update tests that cover the new river workflow behavior, including archive handling and unified spherical river processing.
@xylar xylar force-pushed the add-prepare-river-network branch from 83badd1 to 612e84a Compare May 8, 2026 12:34
@changliao1025
Copy link
Copy Markdown

changliao1025 commented May 8, 2026

Can you provide more info for:

# minimum drainage area to retain in the simplified network (m^2)
drainage_area_threshold = 1.0e10

drainage_area_threshold can be a function of mesh resolution, too.

# minimum separation between retained non-endorheic outlets (m)
outlet_distance_tolerance = 10000.0

outlet_distance_tolerance can be a function of mesh resolution near coast, too.

# minimum tributary-to-main-stem drainage-area ratio for retaining a nearby
# tributary at a confluence
tributary_area_ratio = 0.05

# inland clip distance for base-mesh river products (km)
base_mesh_clip_distance_km = 20.0

# geometry simplification tolerance for base-mesh river products (km)
base_mesh_simplify_tolerance_km = 2.0

# minimum retained segment length after base-mesh clipping (km)
base_mesh_min_segment_length_km = 2.0

# optional minimum outlet stub length to preserve after clipping (km)
base_mesh_preserve_outlet_stub_km = 0.0

@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented May 9, 2026

drainage_area_threshold can be a function of mesh resolution, too.

outlet_distance_tolerance can be a function of mesh resolution near coast, too.

Okay, that's good to know. Can you be more specific about how each of these should be specified as a lat-lon field? How do we know the mesh resolution at the point where we're trying to simplify the river network with this information? At the moment, these config options are used before we have constructed the sizing field, as they are inputs to the sizing field itself.

Update: Claude helped me understand this better. You aren't deriving these from the resolution maps, just from the scalar resolution values. I'll try to figure out how to do something similar in Polaris.

@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented May 9, 2026

Can you provide more info for

I will try to update the user's guide to give a more verbose description of how each of these is used.

This includes moving the coastline docs into a unified mesh
section and some clean up to those docs as well.
@xylar xylar force-pushed the add-prepare-river-network branch from 612e84a to 25f1998 Compare May 9, 2026 09:54
@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented May 9, 2026

@changliao1025, I had Claude expand quite a bit on the descriptions of these config options. Could you take a look here and see if that helps?
https://github.com/xylar/polaris/blob/add-prepare-river-network/docs/users_guide/mesh/tasks/unified/river.md#configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request mesh Related to the mesh component unified-mesh

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants