Verification fixes#205
Conversation
| dir=$RUNROOT/visualization/grid2grid/$extent/plots-spatial-stats | ||
| count=$(find $dir -type f -name "*.png" | wc -l) | ||
| expected=8 | ||
| expected=32 |
There was a problem hiding this comment.
The updated configuration calls for verification to start at the first non-zero leadtime, so for six-hourly forecasts to a max leadtime of 24, we have 4 forecasts (6, 12, 18, 24) x 8 forecast variables = 32 diff plots expected.
| inference_start: !datetime 2022-02-03T12 | ||
| inference_stop: !datetime '{{ app.time.inference_start }}' | ||
| leadtime: 24 | ||
| max_leadtime: 24 |
There was a problem hiding this comment.
I thought this updated name might better reflect the fact that inference produces forecasts at multiple leadtimes, not just one, which, to me, the term leadtime implies. I hope that max_leadtime implies that there are multiple leadtimes.
| start: !int '{{ val.step }}' | ||
| end: !int '{{ app.time.max_leadtime }}' |
There was a problem hiding this comment.
These should be tied to the leadtime range of the overall configuration.
| validtime: time | ||
| leadtimes: | ||
| start: '{{ app.time.leadtime }}' | ||
| start: '{{ app.time.step }}' |
There was a problem hiding this comment.
Verification should start at the first non-zero leadtime.
| meta: | ||
| workdir: '{{ val.vx.rundir }}/TRUTH/EXTENT' | ||
| grids: '{{ val.vx.rundir }}/TRUTH/EXTENT/grids' | ||
| max_leadtime: !int '{{ app.time.max_leadtime }}' |
There was a problem hiding this comment.
Technical detail: This value needs to be copied here from the app.time block so that it is available when wxvx runs and no longer has access to the full configuration but only to one specific wxvx: block.
| forecast: | ||
| name: global | ||
| path: '{{ meta.post_files }}/global.{{ yyyymmdd[:4] }}-{{ yyyymmdd[4:6] }}-{{ yyyymmdd[6:8] }}T{{ hh }}.{{ "%02d" % fh }}h.nc' | ||
| path: '{{ meta.post_files }}/global.{{ yyyymmdd[:4] }}-{{ yyyymmdd[4:6] }}-{{ yyyymmdd[6:8] }}T{{ hh }}.{{ "%02d" % meta.max_leadtime }}h.nc' |
There was a problem hiding this comment.
Here and in several similar diffs that follow, respect the fact that inference writes forecasts for multiple leadtimes to a single netCDF file that includes the max-leadtime value in its filename. This lets wxvx find all of the forecasts in the appropriate, single file.
github-actions
Bot
added
ci-running
mariahpope
left a comment
mariahpope
left a comment
There was a problem hiding this comment.
This looks great, and exactly what I had to do to get things to run... but you did it elegantly instead of hard coding like me :)
Lets see if CI passes, with more plots happening now I want to make sure that the slurm wall clock is enough time (it should be). I only ran into this when doing the full 240 hours and a lot more variables. But, something to double check and CI will tell us.
As long as everything passes I am good to merge this!
@mariahpope CI passed, so I think we're good for now. Someday it would be nice to try to scale wallclock values based on the quantity of work to be done. |
4 participants
Description:
Fixes #185. In manual tests on Ursa, diff plots are no longer blank, there are no errors in the
postwxvx.logfiles, and the expected number of plots are produced.Type of change:
Area(s) affected
Commit Requirements:
Testing / Verification: