USGS NLDAS fails with a floating invalid error

[ekluzek]

The test

SMS_D.nldas2_nldas2_rUSGS_mnldas2.I2000Clm60SpMizGs.derecho_intel.mizuroute-default

in v3.1.1 with ctsm5.4.044

fails with a floating invalid as follows:

cesm.log:

dec0368.hsn.de.hpc.ucar.edu 128:  ROF: PIO root=           1
dec0368.hsn.de.hpc.ucar.edu 128:  shr_file_mod.F90         913
dec0368.hsn.de.hpc.ucar.edu 128:  This routine is depricated - use shr_log_setLogUnit instead        -131
dec0368.hsn.de.hpc.ucar.edu 128:  shr_file_mod.F90         913
dec0368.hsn.de.hpc.ucar.edu 128:  This routine is depricated - use shr_log_setLogUnit instead        -131
dec0371.hsn.de.hpc.ucar.edu 354: forrtl: error (65): floating invalid
dec0372.hsn.de.hpc.ucar.edu 505: forrtl: error (65): floating invalid
dec0371.hsn.de.hpc.ucar.edu 354: Image              PC                Routine            Line        Source
dec0372.hsn.de.hpc.ucar.edu 505: Image              PC                Routine            Line        Source
dec0372.hsn.de.hpc.ucar.edu 505: libc.so.6          0000150E37FCD900  Unknown               Unknown  Unknown
dec0371.hsn.de.hpc.ucar.edu 354: libc.so.6          0000146013551900  Unknown               Unknown  Unknown
dec0372.hsn.de.hpc.ucar.edu 505: cesm.exe           0000000003BA3C83  water_height              196  hydraulic.f90
dec0372.hsn.de.hpc.ucar.edu 505: cesm.exe           0000000003B37798  diffusive_wave            329  dfw_route.f90
dec0371.hsn.de.hpc.ucar.edu 354: cesm.exe           0000000003BA3C83  water_height              196  hydraulic.f90
dec0371.hsn.de.hpc.ucar.edu 354: cesm.exe           0000000003B37798  diffusive_wave            329  dfw_route.f90
dec0372.hsn.de.hpc.ucar.edu 505: cesm.exe           0000000003B32426  dfw_rch                   162  dfw_route.f90

This is after mizuRoute initialization and into the run loop.

I didn't see anything of note in any of the ESMF PET files.

[nmizukami]

I believe I know the root cause of this.

Cause:
GF network netCDF includes one reach, which is a headwater reach without any catchments connected. So the reach width gets zero (reach width is a function of total upstream catchment area). And now new setup puts lateral flow (which is also zero m3/s) at the top of reach and route through the reach for headwater reach, and also tries to compute flow depth (water height), which is computed by dividing reach storage divided by reach width (which is zero). so the code died there (dividing by zero).

To fix:
maybe just removing that headwater reach since this reach get nothing to route, and always zero. This creates new updated network data.

[ekluzek]

I believe I know the root cause of this.

To fix: maybe just removing that headwater reach since this reach get nothing to route, and always zero. This creates new updated network data.

Cool. That's great. So we just need a new USGS ntopo file. That's sounds straightforward to do. I suggest at the same time that a check be added after the ntopo file is read and the reach width is set to check that it's > 0.0 and abort if not. This way the check would be done for any new ntopo dataset and help the user figure out the problem with a graceful error check.

Since, this would require a new ntopo dataset at the same time you could change the _FillValue on the dataset to something besides NaN (to help fix #636).

This sounds pretty reasonable to do, I'll let you decide on the priority for it.