Feature #1518 winds#3403
Merged
Merged
Conversation
…rInfo base class.
…d, used a pre-defined hard-coded list of variable names to drive the derivation logic.
…class. Next up are handling wind rotations and adding lots of tests.
…add long names and units for derived wind fields.
…ing winds, the level string needs to be wrapped in parenthesis based on the existing parsing logic.
…r than just the derived classes for GRIB1 and GRIB2.
…nce. Still need to add the vorticity and divergence derivations and add new unit tests.
… they're non-trivial.
…CF) contain earth-relative winds.
…cp_combine -sum command to work again on precip data where name = APCP_03 while req_name = APCP.
…ptive disc/pcat/pnum.
…). We now have rotate_winds() for both DataPlane and DataPlaneArray inputs. And wind rotation is now handled AFTER the calls to process_data_plane().
…as to make the unit tests happier. ci-run-unit
…umber of them relative to the develop branch.
…into feature_1518_winds
JohnHalleyGotway
marked this pull request as ready for review
June 25, 2026 21:04
…lls to mlog throughout the MET repo.
…hen the current field name setting matches the contents of what's in ConfigConstants. That way, it is NOT used when the user explicitly sets the field name.
…satisfy SonarQube's complaints about calling new and delete.
|
@JohnHalleyGotway, excellent job on stepping through the numerous details and code changes for this PR. I ran through the new unit tests and looked at the log files, tested on some GRIB2 data (against prepbufr data), and I also cross-checked the MET output from this branch against a current set-up using a user script (leveraging wrf-python) to derive the earth-relative wind components and wind speed from WRF output files. All tests looked good and ran and produced output as expected. In the case of comparing MET derivation functionality against wrf-python, no differences in the output were noted. |
michelleharrold
previously approved these changes
Jul 13, 2026
michelleharrold
approved these changes
Jul 14, 2026
michelleharrold
left a comment
michelleharrold
left a comment
There was a problem hiding this comment.
Mods to the documentation look great.
|
github-project-automation
Bot
moved this from 🔎 In review
to 🏁 Done
in METplus-13.0 Development
JohnHalleyGotway
added a commit
that referenced
this pull request
Jul 15, 2026
* Feature 2673 sonarqube beta5 redundant parentheses (#2930) * #2673 Removed redundant_parentheses * #2673 Removed redundant_parentheses * #2673 Removed redundant parentheses * #2673 Removed redundant parentheses --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Fix release checksum action (#2929) * Feature 2857 tripolar coordinates (#2928) * #2857 Added MetNcCFDataFile::build_grid_from_lat_lon_vars * #2857 Added NcCfFile::build_grid_from_lat_lon_vars * #2857 Check the coordinates attribute to find latitude, longitude, and time variables * #2857 Get the lat/lon variables from coordinates attribute if exists * #2857 Added two constants * #2857 Deleted debug messages * #2857 Added lat_vname and lon_vname for var_name_map * #2857 Added two unit tests: point2grid_sea_ice_tripolar and point2grid_sea_ice_tripolar_config * #2857 Initial release * #2857 Correct dictinary to get file_type * #2857 DO not check the time variable for point2grid * #2857 Added point2grid_tripolar_rtofs --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature 2932 v12.0.0-beta5 (#2933) * Per #2932, updating version and release notes * Per #2932, updating date on release notes * Per #2932, fixed formatting and links * Update release-notes.rst * Update release-notes.rst Removing inline backticks since they do not format the way I expected, especially when put inside bolded release notes. --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature fix release notes (#2934) * Fixing up release notes * Update release-notes.rst --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Per dtcenter/METplus#2643 discussion, add more detail about the budget interpolation method. * Feature #2924 fcst climo, PR 1 of 2 (#2939) * Per #2924, Update the MPR and ORANK output line types to just write duplicate existing climo values, update the header tables and MPR/ORANK documentation tables. * Per #2924, update get_n_orank_columns() logic * Per #2924, update the Stat-Analysis parsing logic to parse the new MPR and ORANK climatology columns. * Per #2924, making some changes to the vx_statistics library to store climo data... but more work to come. Committing this first set of changes that are incomplete but do compile. * Per #2924, this big set of changes does compile but make test produces a segfault for ensemble-stat * Per #2924, fix return value for is_keeper_obs() * Per #2924, move fcst_info/obs_info into the VxPairBase pointer. * Per #2924, update Ensemble-Stat to set the VxPairBase::fcst_info pointer * Per #2924 udpate handling of fcst_info and obs_info pointers in Ensemble-Stat * Per #2924, update the GSI tools to handle the new fcst climo columns. * Per #2924, add backward compatibility logic so that when old climo column names are requested, the new ones are used. * Per #2924, print a DEBUG(2) log message if old column names are used. * Per #2924, switch the unit tests to reference the updated MPR column names rather than the old ones. * Per #2924, working progress. Not fully compiling yet * Per #2924, another round of changes. Removing MPR:FCST_CLIMO_CDF output column. This compiles but not sure if it actually runs yet * Per #2924, work in progress * Per #2924, work in progress. Almost compiling again. * Per #2924, get it compiling * Per #2924, add back in support for SCP and CDP which are interpreted as SOCP and OCDP, resp * Per #2924, update docs about SCP and CDP threshold types * Per #2924, minor whitespace changes * Per #2924, fix an uninitialized pointer bug by defining/calling SeepsClimoGrid::init_from_scratch() member function. The constructor had been calling clear() to delete pointers that weren't properly initialized to nullptr. Also, simplify some map processing logic. * Per #2924, rename SeepsAggScore from seeps to seeps_agg for clarity and to avoid conflicts in member function implementations. * Per #2924, fix seeps compilation error in Point-Stat * Per #2924, fix bug in the boolean logic for handling the do_climo_cdp NetCDF output option. * Per #2924, add missing exit statement. * Per #2924, tweak threshold.h * Per #2924, define one perc_thresh_info entry for each enumerated PercThreshType value * Per #2924, simplify the logic for handling percentile threshold types and print a log message once when the old versions are still used. * Per #2924, update the string comparison return value logic * Per #2924, fix the perc thresh string parsing logic by calling ConcatString::startswith() * Per #2924, switch all instances of CDP to OCDP. Gen-Ens-Prod was writing NetCDF files with OCDP in the output variable names, but Grid-Stat was requesting that the wrong variable name be read. So the unit tests failed. * Per #2924, add more doc details * Per #2924, update default config file to indicate when climo_mean and climo_stdev can be set seperately in the fcst and obs dictionaries. * Per #2924, update the MET tools to parse climo_mean and climo_stdev separately from the fcst and obs dictionaries. * Per #2924, backing out new/modified columns to minimize reg test diffs * Per #2924, one more section to be commented out later. * Per #2924, replace several calls to strncmp() with ConcatString::startswith() to simplify the code * Per #2924, strip out some more references to OBS_CLIMO_... in the unit tests. * Per #2924, delete accidental file * Per #2924 fix broken XML comments * Per #2924, fix comments * Per #2924, address SonarQube findings * Per #2924, tweak a Point-Stat and Grid-Stat unit test config file to make the output more comparable to develop. * Per #2924, fix bug in the logic of PairDataPoint and PairDataEnsemble, when looping over the 3-dim array do not return when checking the climo and fcst values. Instead we need to continue to the next loop iteration. * Per #2924, address more SonarQube code smells to reduce the overall number in MET for this PR. * Per #2924, correct the logic for parsing climo data from MPR lines. * Per #2924, cleanup grid_stat.cc source code by making calls to DataPlane::is_empty() and Grid::nxy(). * Per #2924, remove unneeded ==0 * Hotfix to the develop branch for a copy/paste bug introduced by PR #2939 * Feature #2924 sal1l2_mae, PR 3 of 3 (#2943) * Per #2924, track SL1L2 and SAL1L2 MAE scores with separate variables since they are no longer the same value. I renamed the existing 'mae' as 'smae' and added a new 'samae' variable. Renaming the existing lets me use the compiler help find all references to it throughout the code. * Per #2924, update the User's Guide climatology details and equations. * Per #2924, some changes to aggr_stat_line.cc and series_analysis.cc to satisfy some SonarQube code smells. * Update develop to clarify masking poly options based on METplus Discussion dtcenter/METplus#2650 * Remove two semi-colons that are not actually necessary to avoid confusion. * Per dtcenter/METplus#2653 discussion, update the MTD usage statement to clarify that data specified in the fcst dictionary is read from the -single input files. * Feature #2924 fcst climo, PR 2 of 3 (#2942) * Per #2924, Update the MPR and ORANK output line types to just write duplicate existing climo values, update the header tables and MPR/ORANK documentation tables. * Per #2924, update get_n_orank_columns() logic * Per #2924, update the Stat-Analysis parsing logic to parse the new MPR and ORANK climatology columns. * Per #2924, making some changes to the vx_statistics library to store climo data... but more work to come. Committing this first set of changes that are incomplete but do compile. * Per #2924, this big set of changes does compile but make test produces a segfault for ensemble-stat * Per #2924, fix return value for is_keeper_obs() * Per #2924, move fcst_info/obs_info into the VxPairBase pointer. * Per #2924, update Ensemble-Stat to set the VxPairBase::fcst_info pointer * Per #2924 udpate handling of fcst_info and obs_info pointers in Ensemble-Stat * Per #2924, update the GSI tools to handle the new fcst climo columns. * Per #2924, add backward compatibility logic so that when old climo column names are requested, the new ones are used. * Per #2924, print a DEBUG(2) log message if old column names are used. * Per #2924, switch the unit tests to reference the updated MPR column names rather than the old ones. * Per #2924, working progress. Not fully compiling yet * Per #2924, another round of changes. Removing MPR:FCST_CLIMO_CDF output column. This compiles but not sure if it actually runs yet * Per #2924, work in progress * Per #2924, work in progress. Almost compiling again. * Per #2924, get it compiling * Per #2924, add back in support for SCP and CDP which are interpreted as SOCP and OCDP, resp * Per #2924, update docs about SCP and CDP threshold types * Per #2924, minor whitespace changes * Per #2924, fix an uninitialized pointer bug by defining/calling SeepsClimoGrid::init_from_scratch() member function. The constructor had been calling clear() to delete pointers that weren't properly initialized to nullptr. Also, simplify some map processing logic. * Per #2924, rename SeepsAggScore from seeps to seeps_agg for clarity and to avoid conflicts in member function implementations. * Per #2924, fix seeps compilation error in Point-Stat * Per #2924, fix bug in the boolean logic for handling the do_climo_cdp NetCDF output option. * Per #2924, add missing exit statement. * Per #2924, tweak threshold.h * Per #2924, define one perc_thresh_info entry for each enumerated PercThreshType value * Per #2924, simplify the logic for handling percentile threshold types and print a log message once when the old versions are still used. * Per #2924, update the string comparison return value logic * Per #2924, fix the perc thresh string parsing logic by calling ConcatString::startswith() * Per #2924, switch all instances of CDP to OCDP. Gen-Ens-Prod was writing NetCDF files with OCDP in the output variable names, but Grid-Stat was requesting that the wrong variable name be read. So the unit tests failed. * Per #2924, add more doc details * Per #2924, update default config file to indicate when climo_mean and climo_stdev can be set seperately in the fcst and obs dictionaries. * Per #2924, update the MET tools to parse climo_mean and climo_stdev separately from the fcst and obs dictionaries. * Per #2924, backing out new/modified columns to minimize reg test diffs * Per #2924, one more section to be commented out later. * Per #2924, replace several calls to strncmp() with ConcatString::startswith() to simplify the code * Per #2924, strip out some more references to OBS_CLIMO_... in the unit tests. * Per #2924, delete accidental file * Per #2924 fix broken XML comments * Per #2924, fix comments * Per #2924, address SonarQube findings * Per #2924, tweak a Point-Stat and Grid-Stat unit test config file to make the output more comparable to develop. * Per #2924, fix bug in the logic of PairDataPoint and PairDataEnsemble, when looping over the 3-dim array do not return when checking the climo and fcst values. Instead we need to continue to the next loop iteration. * Per #2924, address more SonarQube code smells to reduce the overall number in MET for this PR. * Per #2924, correct the logic for parsing climo data from MPR lines. * Per #2924, update MPR and ORANK line types to update/add FCST/OBS_CLIMO_MEAN/STDEV/CDF columns. * Per #2924, cleanup grid_stat.cc source code by making calls to DataPlane::is_empty() and Grid::nxy(). * Per #2924, remove unneeded ==0 * Per #2924, working on PR2. * Per #2924, update User's Guide with notional example of specifying climo_mean and climo_stdev separately in the fcst and obs dicts. * Per #2924, adding a new unit test. It does NOT yet run as expected. Will debug on seneca * Per #2924, pass the description string to the read_climo_data_plane*() function to provide better log messages * Per #2924, more work on consistent log messages * Per #2924, tweak the configuration to define both field, climo_mean, and climo_stdev in both the fcst and obs dictionaries * Per #2924, tweak the unit_climatology_mixed.xml test * Per #2924, only whitespace changes. * Per #2924, missed swapping MET #2924 changes in 3 test files * Per #2924, delete accidentally committed file * Per #2924, delete accidentally committed files * Per #2924, add support for GRIB1 time range indicator value of 123 used for the corresponding METplus Use Case. Note that there are 22 other TRI values not currently supported. * Adds caveat regarding longitudes appearing in DEBUG statements with a… (#2947) * Adds caveat regarding longitudes appearing in DEBUG statements with a different sign to the FAQ. * Update appendixA.rst Missing paren * Create install_met_env.cactus * Adding special script for installing beta5 on wcoss2 * Modifying script, including updates to eckit and atlas * Corrected version of bufr being used * Feature #2938 pb2nc_center_time (#2954) * Per #2938, define CRC_Array::add_uniq(...) member function which is now used in PB2NC * Per #2938, replace n_elements() with n() to make the code more concise. Refine log/warning message when multiple message center times are encountered. * Feature #1371 series_analysis (#2951) * Per #1371, add -input command line argument and add support for ALL for the CTC, MCTC, SL1L2, and PCT line types. * Per #1371, rename the -input command line option as -aggregate instead * Per #1371, work in progress * Per #1371, just comments * Per #1371, working on aggregating CTC counts * Per #1371, work in progress * Per #1371, update timing info using time stamps in the aggr file * Per #1371, close the aggregate data file * Per #1371, define set_event() and set_nonevent() member functions * Per #1371, add logic to aggregate MCTC and PCT counts * Merging changes from develop * Per #1371, work in progress aggregating all the line statistics types. Still have several issues to address * Per #1371, switch to using get_stat() functions * Per #1371, work in progress. More consolidation * Per #1371, correct expected output file name * Per #1371, consistent regridding log messages and fix the Series-Analysis PairDataPoint object handling logic. * Per #1371, check the return status when opening the aggregate file. * Per #1371, fix prc/pjc typo * Per #1371, fix the series_analysis PCT aggregation logic and add a test to unit_series_analysis.xml to demonstrate. * Per #1371, resolve a few SonarQube findings * Per #1371, make use of range-based for loop, as recommeded by SonarQube * Per #1371, update series-analysis to apply the valid data threshold properly using the old aggregate data and the new pair data. * Per #1371, update series_analysis to buffer data and write it all at once instead of storing data value by value for each point. * Per #1371, add useful error message when required aggregation variables are not present in the input -aggr file. * Per #1371, print a Debug(2) message listing the aggregation fields being read. * Per #1371, correct operator+= logic in met_stats.cc for SL1L2Info, VL1L2Info, and NBRCNTInfo. The metadata settings, like fthresh and othresh, were not being passed to the output. * Per #1371, the DataPlane for the computed statistics should be initialized to a field of bad data values rather than the default value of 0. Otherwise, 0's are reported for stats a grid points with no data when they should really be reported as bad data! * Per #1371, update logic of the compute_cntinfo() function so that CNT statistics can be derived from a single SL1L2Info object containing both scalar and scalar anomaly partial sums. These changes enable CNT:ANOM_CORR to be aggregated in the Series-Analysis tool. * Per #1371, fix logic of climo log message. * Per #1371, this is actually related to MET #2924. In compute_pctinfo() used obs climo data first, if provided. And if not, use fcst climo data. * Per #1371, fix indexing bug (+i instead of +1) when check the valid data count. Also update the logic of read_aggr_total() to return a count of 0 for bad data. * Per #1371, add logic to aggregate the PSTD BRIERCL and BSS statistics in the do_climo_brier() function. Tested manually to confirm that it works. * Per #1371, switch to using string literals to satisfy SonarQube * Per #1371, update series_analysis tests in unit_climatology_1.0deg.xml to demonstrate aggregating climo-based stats. * Per #1371, remove extra comment * Per #1371, skip writing the PCT THRESH_i columns to the Series-Analysis output since they are not used * Per #1371, fix the R string literals to remove \t and \n escape sequences. * Per #1371, update the read_aggr_data_plane() suggestion strings. * Per #1371, ignore unneeded PCT 'THRESH_' variables both when reading and writing ALL PCT columns. * Per #1371, update the test named series_analysis_AGGR_CMD_LINE to include data for the F42 lead time that had previously been included for the same run in the develop branch. Note however that the timestamps in the output file for the develop branch (2012040900_to_2012041100) were wrong and have been corrected here (2012040900_to_2012041018) to match the actual data. * Per #1371, update the -aggr note to warn users about slow runtimes * Feature 2948 cxx17 (#2953) * Per #2948, updating versions of ecbuild, eckit, and atlas * Per #2948, Adding MET_CXX_STANDARD * Per #2948, updated wording for MET_CXX_STANDARD description * Per #2948, updating script to work with two versions of ecbuild, eckit, and atlas * Per #2948, without this change, there are compilation problems if the user wants to compile wihtout python * Per #2948, fixing logic for MET_CXX_STANDARD * Per #2928, adding missing end bracket * Per #2948, fixed the logic for compiling versions of ecbuild, eckit, and atlas * Per 948, fixed syntax for setting CXXFLAGS * Per #2948, adding new Makefile.in files and configure and changing METbaseimage 3.2 to 3.3. * Per #2948, updating version of met base tag from 3.2 to 3.3 * Per #2948, adding --enable-all MET_CXX_STANDARD=11 job * Update compilation_options.yml * Per #2948, added a job10 for MET_CXX_STANDARD=14 * Per #2948, added brief documentation for the MET_CXX_STANDARD option --------- Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1729 set attr grid (#2955) * #1729 Allow to change to differnt grid size if the raw size is 0 * Added build_grid_by_grid_string and build_grid_by_grid_string * #1729 Calls build_grid_by_grid_string * #1729 Added set_attr_grid at the -field option * #1729 Set obs_type to TYPE_NCCF if the file_type is given at the config file * #1729 Support set_sttr_grid and changed Error messages to Warning * #1729 FIxed SonmarQube findings * #1729 Initial release for unit test * #1729 Added update_missing_values * #1729 Deleted a shadowed local variable * #2673 Added more is_eq * #2673 Added get_exe_duration * 2673 Reducded nested statements * 2673 Fixed SonarGube findings * 2673 Fixed SonarQube findings * 2673 Fixed SonarQube findings * #1729 Added aan unittest plot_data_plane_set_attr_grid * #1729 Added aan unittest point2grid_cice_set_attr_grid * #1729 Added changed back the verbose level * #1729 Corrected typo --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2958 develop BAGSS SEDI CI (#2959) * Bugfix 2936 point2grid gfs (#2964) * #2936 Support 1D lat/lon values * #2936 Initial release * #2936 Cast the data type to avoid a compile warning * #2936 Added an unittest point2grid_gfs_1D_lat_lon --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2968 point2grid set attr grid (#2969) * #2968 Corrected set_attr_grid for point2grid_cice_set_attr_grid * #2968 Compare the DataPlane size and the variable data size * #2968 nx and ny are not ignored with set_attr_grid * #2968 Compare the DataPlane size and the variable data size --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature 2937 update unit (#2944) * added single quotes around env var/val pairs in export statements in cmd only mode * updated logic in unit() to check exec return value against expected return value; created TEST xml file to test this feature * deleted TEST_ xml, added test with retval 1 to unit_ascii2nc --------- Co-authored-by: Natalie Babij <nbabij@seneca.rap.ucar.edu> * Feature #2887 categorical weights PR 1 of 2 (#2967) * Per #2887, update NumArray::vals() to return a reference to the vector rather a pointer to doubles. * Per #2887, switch over the whole ContingencyTable class heirarchy from storing integer counts to storing double-precision weights. * Add ContingencyTable::is_integer() member function to check whether the table contains all integers * Per #2887, update parse_stat_line.cc to get it to compile after changing PCT to store thresholds in a std::vector. * Per #2887, update PCTInfo::clear() logic. * Per #2887, update ctc_by_row() logic to create reproducible results with the develop branch. * Per #2887, update logic of define_prob_bins() to add a final >=1.0 threshold if needed. While ==0.1 works fine, I found that ==0.05 did not because the last >=1.0 threshold was missing likely do to floating point precision issues. This change should fix that problem. * Per #2887, update roc_auc() function to match the develop branch * Per #2887, fix bug if computation of far() * Per #2887, replaced all ==0 integer equality checks with calls to is_eq() instead and fix a couple of equations to snuff out diffs in some CTS statistics. * Per #2887, address some of the 34 SonarQube code smells flagged for this PR. Note that the compute_ci.h/.cc changes are necessary and good since we should be computing CI's using doubles instead of integer counts. * Per #2887, update run_sonarqube.sh to specify the target CXX standard as 11. The hope is that that will limit the findings to only those features available in the C++11 standard. * Per #2887, update to SonarQube version 6.1.0.4477 released on 6/27/2024. * Per #2887, updating build_met_sonarqube.sh to specify --std=c++11 since c++17 is used by default * Hotfix to develop to fix a bug introduced for MET #2887. Refine the define_prob_bins() utility function so that ==n probability thresholds result in the correct number of probability thresholds. We were adding an unncessary 10-th bin (from 1.07143 to 1.0) for the ==7 probability threshold type. * Fix typo in tc-pairs.rst * Update build_docker_and_trigger_metplus.yml The docs directory was moved up to the top-level of the repository but this workflow was not updated. Changing the ignore setting so that doc-only updates do not trigger the full METplus testing workflow. * Feature 2023 remove double quotes around keywords (#2974) * testing AREA and AUTO changes * Keywords B thru L * thru R * adding quotes back in for lower case items * S thru the end of the document * Removing double quotes around 3 key words * Per #2023, adding a label name for the Attributes section * Per #2023, adding an internal link for the MODE tool Attributes section. * Adding quotes around Valid basins entries * more double quote updates * more complex updates with Julie P help * removing double quotes * fixing typos * removing double quotes * unbolding SURFACE and putting it in double quotes * fixing grammar * grammar * fixing typo * fixing typo --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Feature #2924 parse_config (#2963) * Per #2924, remove GenEnsProd config file comment about parsing desc separately from each obs.field entry because the obs dictionary does not exist in the GenEnsProd config file. * Per #2924, update list of needed config entry names * Per #2924, remove const from the parent() member function so that we can perform lookups for the parent. * Per #2924, update the signature for and logic of the utility functions that retrieve the climatology data. Rather than requiring all the climo_mean and climo_stdev dictionary entries to be defined at the same config file context level, parse each one individually. This enables the METplus wrappers to only partially override this dictionary and still rely on the default values provided in MET's default configuration files. * Per #2924, update all calls to the climatology utility functions based on the new function signature. Also update the tools to check the number of climo fields separately for the forecast and observation climos. * Per #2924, update the parsing logic for the climatology regrid dictionary. Use config.fcst.climo_mean.regrid first, config.fcst.regrid second, and config.climo_mean.regrid third. Notably, DO NOT use config.regrid. This is definitely the problem with having regrid specified at mutliple config file context levels. It makes the logic for which to use when very messy. * Per #2924, forgot to add an else to print an error * Per #2924, remove extraneous semicolon * Per #2924, move 'fcst.regrid' into 'fcst.climo_mean.regrid'. Defining the climatology regridding logic inside fcst is problematic because it applies to the forecast data as well and you end up with the verification grid being undefined. So the climo regridding logic must be defined in 'climo_mean.regrid' either within the 'fcst' and 'obs' dictionaries or at the top-level config context. * Per #2924, based on PR feedback from @georgemccabe, add the Upper_Left, Upper_Right, Lower_Right, and Lower_Left interpolation methods to the list of valid options for regridding, as already indicated in the MET User's Guide. * Per #2924, update the logic of parse_conf_regrid() to (hopefully) make it work the way @georgemccabe expects it to. It now uses pointers to both the primary and default dictionaries and parses each entry individually. * Per #2924, need to check for non-null pointer before using it * Per #2924, revise the climo_name dictionary lookup logic when parsing the regrid dictionary. * Per #2924, update logic for handling RegridInfo * Per #2924, remove the default regridding information from the 'Searching' log message to avoid confusion. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2924 parse_config PR 2 (#2975) * Per #2924, remove GenEnsProd config file comment about parsing desc separately from each obs.field entry because the obs dictionary does not exist in the GenEnsProd config file. * Per #2924, update list of needed config entry names * Per #2924, remove const from the parent() member function so that we can perform lookups for the parent. * Per #2924, update the signature for and logic of the utility functions that retrieve the climatology data. Rather than requiring all the climo_mean and climo_stdev dictionary entries to be defined at the same config file context level, parse each one individually. This enables the METplus wrappers to only partially override this dictionary and still rely on the default values provided in MET's default configuration files. * Per #2924, update all calls to the climatology utility functions based on the new function signature. Also update the tools to check the number of climo fields separately for the forecast and observation climos. * Per #2924, update the parsing logic for the climatology regrid dictionary. Use config.fcst.climo_mean.regrid first, config.fcst.regrid second, and config.climo_mean.regrid third. Notably, DO NOT use config.regrid. This is definitely the problem with having regrid specified at mutliple config file context levels. It makes the logic for which to use when very messy. * Per #2924, forgot to add an else to print an error * Per #2924, remove extraneous semicolon * Per #2924, move 'fcst.regrid' into 'fcst.climo_mean.regrid'. Defining the climatology regridding logic inside fcst is problematic because it applies to the forecast data as well and you end up with the verification grid being undefined. So the climo regridding logic must be defined in 'climo_mean.regrid' either within the 'fcst' and 'obs' dictionaries or at the top-level config context. * Per #2924, based on PR feedback from @georgemccabe, add the Upper_Left, Upper_Right, Lower_Right, and Lower_Left interpolation methods to the list of valid options for regridding, as already indicated in the MET User's Guide. * Per #2924, update the logic of parse_conf_regrid() to (hopefully) make it work the way @georgemccabe expects it to. It now uses pointers to both the primary and default dictionaries and parses each entry individually. * Per #2924, need to check for non-null pointer before using it * Per #2924, revise the climo_name dictionary lookup logic when parsing the regrid dictionary. * Per #2924, update logic for handling RegridInfo * Per #2924, remove the default regridding information from the 'Searching' log message to avoid confusion. * Per #2924, escape sequences, like \n, cannot be used inside R-string literals. * Per #2924, update the logic of check_climo_n_vx() * Per #2924, revise logic in read_climo_data_plane_array(). Check the number of climo fields provided. If there's 0, just return since no data has been requested. If there's 1, use it regardless of the number of input fields. If there's more than 1, just use the requested i_vx index value. * Per #2924, update Series-Analysis to set both i_fcst and i_obs when looping over the series entries. * Per #2924, no real change. Just whitespace. * Unrelated to #2924, superficial changes to formatting of method_name strings for consistency. * Per #2924, add a new series_analysis test that ERRORS OUT prior to this PR but works after the changes in this PR. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2949 cxx11 doc (#2973) * Per #2949, updating installation instructions * Per #2949, adding missing colon from note directive * Per #2949, third attempt to get the new note to show up * Per #2949, modifying text and format * Per #2949, removing images in favor of code blocks for easier modification * Per #2949, modified wording for clarity * Per #2929, corrected typo * Update installation.rst No changes to content, only whitespace for consistency, mostly removing tabs. * Update docs/Users_Guide/installation.rst Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Per #2949, testing variable replacement, expect failures * Per #2949, reverting to orignal state after testing --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Bugfix #2979 develop MTD Grid (#2981) * Per #2979, remove nc_grid.h/.cc and replace it with calls to the read_netcdf_grid(...) and write_netcdf_proj(...) library utility functions. Note that these changes do compile but I haven't tested whether they actually fix the underlying problem. Also note that nc_utils_local.h/.cc can also likely be replaced with calls to common library functions. * Per #2979, remove references to nc_grid.o from the MTD test code. * Per #2979, insert a newline in unit.py output between the env vars and the command. * Per #2979, insert a newline in unit.py output between the env vars and the command. * Per #2979, the write_netcdf_proj(...) utility function adds the lat and lon dimensions. Update mtd to NOT define those dimensions prior to calling write_netcdf_proj(...). * Per #2979, minor changes to is_eq() calls to fix compiler warning messages * Per #2979, for the develop branch, also replace nc_utils_local.h/.cc with calls to common library code. Also remove commented out code. * Per #2979, delete commented out code and make error/warning message formatting consistent. * Fixes for SonarQube --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2880 point2grid qc (#2984) * Per #2880, remove Point2Grid quality_mark_thresh config option and add obs_quality_inc and obs_quality_exc from the config file. * Per #2880, update point2grid docs and reformat whitespace throughout. * Per #2880, remove quality_mark_thresh and add obs_quality_inc and obs_quality_exc. * Unrelated to #2880, fix formatting of this R-string which cannot include any special formatting, such as \n. * Per #2880, not working quite right yet but this is progress * Per #2880, add a write_css(IntArray) utility function. * Per #2880, update NcPointObsData class to read the obs quality values from the input file. * Per #2880, update the log message about the quality control filter options applied. * Per #2880, tweak log messages. * Per #2880, tweak log messages. * Per #2880, add a Point2Grid unit test to demonstrate using the obs_quality_inc and obs_quality_exc options. * Per #2880, declare these get accessor functions as const to avoid SonarQube code smell. * Per #2880, many changes to the vx_nc_obs library and point2grid application to replace dynamically allocated memory with STL vectors to satisfy SonarQube code smells. * Per #2880, fix compilation error. * Per #2880, fix compilation error * Per #2880, revert skip_times back to vector<bool> since I wasn't postive the casting (int *) to (bool *) would actually work in the way I expect. This is safer. * Per #2880, rename the -qc command line option as -goes_qc, but still quietly support -qc * Per #2880, based on guidance from @hsoh, reset the var_cell_mapping vector for each loop iteration. * Spelling correction * Feature #2882 seeps qa (#2987) * Update seeps.h Change variable names to reduce ambiguity for interpretation and aid useability. * Update seeps.cc Pull through variable name changes and renaming of functions to aid legibility and clarity. Introduced some additional debug print statements. * Update grid-stat.rst Add documentation about the location of the gridded climatology files for SEEPS and which environment variable to use. * Replace read_seeps_scores() with get_seeps_climo_grid() * Manually merging Rachel's patch-1 changes. * Getting close to getting these seeps changes to compile. But it's failing in pair_data_point.cc * Per #2882, get branch feature_2882_seeps_qa compiling again. Recommend revisiting the volume of SEEPS-related Debug log messages and reducing them once its fully tested. * Per #2882, need to update the handling of the PPT24_seepsweights_grid.nc file name. Rename as _v12.0.nc for the updated version with the new names so that the existing regressions tests and nightly builds for main_v11.1 and develop continue to work. We can remove the _v12.0 once this feature branch is merged into develop but for the time being, we need both versions to exist. * Per #2882, rename the SEEPS columns from S12, S13, S21, S23, S31, S32 to the more descriptive ODFL, ODFH, OLFD, OLFH, OHFD, OHFL names. * Per #2882, update SEEPS details * Per #2882, store and report the weighted mean fcst and mean obs, just like the SEEPS score itself so that they're handled in a consistent manner. Note however that it's hard-coded to NOT write the weighted means/score, only the unweighted ones. * Per #2882, change SEEPS debug log levels and correct the storage of mean_fcst and mean_obs values. * Per #2882, correct SEEPS column name lookups * Per #2882, call is_bad_data() instead of is_eq(..., -9999.0) to get rid of compiler warning message. * Per #2882, add 2 more variations of the is_eq() function with mixed float and double inputs to satisfy compiler pb2nc compiler warnings. * Per #2882, switch from dynamically allocated arrays to std::vector * Per #2882, enhance Stat-Analysis to write the SEEPS line type to an output .stat file. * Per #2882, update the aggregated seeps computation to use better-initialized vectors. * Per #2882, resolve a few more SonarQube code smells. * Per #2882, now that this PR is ready to merge, remove the v12.0 version number from the gridded SEEPS climo file name ci-skip-all --------- Co-authored-by: mpm-meto <64001904+mpm-meto@users.noreply.github.com> * Hotfix to the develop branch for unit_grid_stat.xml to use the updated gridded seeps climo file name. * Feature #2887 categorical weights (#2988) * Per #2887, update NumArray::vals() to return a reference to the vector rather a pointer to doubles. * Per #2887, switch over the whole ContingencyTable class heirarchy from storing integer counts to storing double-precision weights. * Add ContingencyTable::is_integer() member function to check whether the table contains all integers * Per #2887, update parse_stat_line.cc to get it to compile after changing PCT to store thresholds in a std::vector. * Per #2887, update PCTInfo::clear() logic. * Per #2887, update ctc_by_row() logic to create reproducible results with the develop branch. * Per #2887, update logic of define_prob_bins() to add a final >=1.0 threshold if needed. While ==0.1 works fine, I found that ==0.05 did not because the last >=1.0 threshold was missing likely do to floating point precision issues. This change should fix that problem. * Per #2887, update roc_auc() function to match the develop branch * Per #2887, fix bug if computation of far() * Per #2887, replaced all ==0 integer equality checks with calls to is_eq() instead and fix a couple of equations to snuff out diffs in some CTS statistics. * Per #2887, address some of the 34 SonarQube code smells flagged for this PR. Note that the compute_ci.h/.cc changes are necessary and good since we should be computing CI's using doubles instead of integer counts. * Per #2887, update run_sonarqube.sh to specify the target CXX standard as 11. The hope is that that will limit the findings to only those features available in the C++11 standard. * Per #2887, update to SonarQube version 6.1.0.4477 released on 6/27/2024. * Per #2887, updating build_met_sonarqube.sh to specify --std=c++11 since c++17 is used by default * Per #2887, swap in a much simpler implementation of the ORSS statistic to match the equation listed in the MET User's Guide. * Per #2887, update grid_stat and library code to actually apply the grid_weight_flag settings to the computation of contingency table counts and statistics. * Per #2887, fix the handling of bad data in the ORSS equation. * Per #2887, add Npairs member to the ContingencyTable class, eliminate the n() accessor function, and carefully replace references to n() with n_pairs() for the integer number of matched pairs or total() with the double-precision sum of the weights. * Per #2887, reset Npairs = 0 for ContingencyTable::zero_out() * Per #2883, need to call set_n_pairs() in a few spots to set ECLV TOTAL column correctly ci-run-unit * Per #2887, call set_n_pairs() when aggregating PCT data in Series-Analysis ci-run-unit * Per #2887, update stat_analysis to parse the TOTAL column for the PCT and MCTC line types. * Pet #2882, call set_n_pairs() after set_size() ci-run-unit * Per #2887, reconfigure existing Ensemble-Stat unit test to request probabilistic output to see that it's impacted by the grid_weight_flag setting. * Per #2887, update Ensemble-Stat test to provide climo stdev data * Per #2887, add grid_weight_flag to the list of config options for Grid-Stat and Ensemble-Stat. * Per #2887, disable FHO output if grid_weight_flag != NONE. * Per #2887, revise the existing unit_grid_weight.xml unit tests for Grid-Stat to write CTC/CTS/MCTC/MCTS output and for the DESC column to be populated to indicate the type of grid weighting that was applied. * Per #2887, relatively small changes to drive down SonarQube code smells. Also, switch from total() to n_pairs() when computing confidence intervals. * Per #2887, more SonarQube tweaks * Per #2887, more SonarQube tweaks. * Per #2887, more SonarQube tweaks. * Per #2887, whitespace only changes. * Per #2287, fix path the seeps climo grid. * Per #2887, update the grid_weight_flag documentation. * Per #2887, tweak the wording. * Feature 2673 sonarqube beta6 vector (#2991) * 673 Replaceddynamic allocatin method (using vector instead of new) * 673 SonarQube findings: use vector for the dynamic memory allocation * 673 SonarQube findings: use vector for the dynamic memory allocation * #2673 Check None of met_data.shape * #2673 Check None of met_data.shape --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature #2279 point_weight_flag (#2993) * Per #2887, update NumArray::vals() to return a reference to the vector rather a pointer to doubles. * Per #2887, switch over the whole ContingencyTable class heirarchy from storing integer counts to storing double-precision weights. * Add ContingencyTable::is_integer() member function to check whether the table contains all integers * Per #2887, update parse_stat_line.cc to get it to compile after changing PCT to store thresholds in a std::vector. * Per #2887, update PCTInfo::clear() logic. * Per #2887, update ctc_by_row() logic to create reproducible results with the develop branch. * Per #2887, update logic of define_prob_bins() to add a final >=1.0 threshold if needed. While ==0.1 works fine, I found that ==0.05 did not because the last >=1.0 threshold was missing likely do to floating point precision issues. This change should fix that problem. * Per #2887, update roc_auc() function to match the develop branch * Per #2887, fix bug if computation of far() * Per #2887, replaced all ==0 integer equality checks with calls to is_eq() instead and fix a couple of equations to snuff out diffs in some CTS statistics. * Per #2887, address some of the 34 SonarQube code smells flagged for this PR. Note that the compute_ci.h/.cc changes are necessary and good since we should be computing CI's using doubles instead of integer counts. * Per #2887, update run_sonarqube.sh to specify the target CXX standard as 11. The hope is that that will limit the findings to only those features available in the C++11 standard. * Per #2887, update to SonarQube version 6.1.0.4477 released on 6/27/2024. * Per #2887, updating build_met_sonarqube.sh to specify --std=c++11 since c++17 is used by default * Per #2887, swap in a much simpler implementation of the ORSS statistic to match the equation listed in the MET User's Guide. * Per #2887, update grid_stat and library code to actually apply the grid_weight_flag settings to the computation of contingency table counts and statistics. * Per #2887, fix the handling of bad data in the ORSS equation. * Per #2887, add Npairs member to the ContingencyTable class, eliminate the n() accessor function, and carefully replace references to n() with n_pairs() for the integer number of matched pairs or total() with the double-precision sum of the weights. * Per #2887, reset Npairs = 0 for ContingencyTable::zero_out() * Per #2883, need to call set_n_pairs() in a few spots to set ECLV TOTAL column correctly ci-run-unit * Per #2887, call set_n_pairs() when aggregating PCT data in Series-Analysis ci-run-unit * Per #2887, update stat_analysis to parse the TOTAL column for the PCT and MCTC line types. * Pet #2882, call set_n_pairs() after set_size() ci-run-unit * Per #2887, reconfigure existing Ensemble-Stat unit test to request probabilistic output to see that it's impacted by the grid_weight_flag setting. * Per #2887, update Ensemble-Stat test to provide climo stdev data * Per #2887, add grid_weight_flag to the list of config options for Grid-Stat and Ensemble-Stat. * Per #2887, disable FHO output if grid_weight_flag != NONE. * Per #2887, revise the existing unit_grid_weight.xml unit tests for Grid-Stat to write CTC/CTS/MCTC/MCTS output and for the DESC column to be populated to indicate the type of grid weighting that was applied. * Per #2279, add the MaskSID struct to store information about station id names and corresponding weights. * Per #2279, add new PointWeightType enumeration along with code to parse it. * Per #2279, adding point_weight_flag option to all Point-Stat and Ensemble-Stat config file and tweaking whitespace. * Per #2279, add point_weight_flag to the Point-Stat and Ensemble-Stat config class. Also remove sue unneeded wgt_dp argument for the add_point_obs() functions. Plan to add logic to set the point weights only AFTER all the observations have been collected for each verification task. * Per #2279, use the default_weight contstant instead of the literal 1.0 value. * Per #2279, add stubs for actually applying the point_weight_flag settings. * Per #2279, fix PairBase to actually set point weight values parsed from station id masks. * Per #2279, trying to fix 2 sonarqurqube bugs * Per #2279, fix a couple bugs parsing the SID weights and add a new unit_point_weight.xml unit test to run Point-Stat on scalar and probability inputs weighting the stations by their elevation. Still need to add Ensemble-Stat calls. * Per #2279, fix small bug ci-run-unit * Per #2279, add ensemble_stat calls to unit_point_weight.xml * Per #2279, add documentation about the point_weight_flag configuration option. * Per #2279, working on debug and warning messages. * Per #2279, tweak the user's guide * Per #2279, switch MaskSID::sid_list from a vector of pairs to a simpler map named sid_map. * Per #2279, fix the madis2nc call to parse_sid_mask() * Per #2279, move MaskSID from vx_config over into dedicated vx_util/mask_sid.h and .cc to be consistent with mask_poly.h. I note that the members of the MaskSID struct were not being initialized properly. So making it a complete class was the right solution. * Per #2279, another change to make it compile. * Per #2279, more tweaks to get it to compile. * Feature #2893 output_obtype (#2995) * Per #2893, add obtype_as_obs_message_type_flag option for Point-Stat and Ensemble-Stat that applies when writing MPR, SEEPS_MPR, and ORANK line types. If true, write the input observation message type to the OBTYPE output column rather than the name of the group. * Per #2893, add option to default config files and fix compilation errors. * Per #2893, change the name from obtype_as_obs_message_type_flag to obtype_as_group_val_flag. * Per #2893, add docs ci-run-unit * Per #2893, add obtype_as_group_val_flag entry for all config files that already include the message_type_group_map entry. * Per #2893, modify existing PointStatConfig_WINDS config file to have it write MPR output with obtype_as_group_val_flag = TRUE. Note that I switched the message types from ADPSFC, SFCSHP, USERSF to just USERSF. The original reason for the 3 was to prove that the counts for ADPSFC + SFCSHP = USERSF. * Per #2893, update commit history for ensemble_stat and point_stat * Per #2893, revert back PointStatConfig_WINDS to minimize diffs * Per #2893, add a Point-Stat unit test to demonstrate setting the obtype_as_group_val_flag option. * Per #2893, need to reset the obtype after writing all MPR, SEEPS_MPR, and ORANK line types in Point-Stat * Per #2893, tweak Point-Stat configuration to also write HIRA ORANK and PCT outputs. * Feature 2673 sonarqube beta6 vector dictionary (#2994) * #2673 SonarQube findings: use vector for the dynamic memory allocation * #2673 Check if a vector is allocated * #2673 Check if a vector is allocated * #2673 SonarQube findings use vector * #2673 Resolved SonarQube findingds * #2673 SonarQube: Replaced f-strings to regular strings * #2673 Added staticmethod to read_mpr * #2673 SonarQube: removed unnecessary pass * #2673 SonarQube: use vector * #2673 Initialize vecvgtor variables * 673 Check if an user python command is empty * #2673 Changed 0 to nullptr * #2673 Rollback for s and av * #2673 Applied the static method call * #2673 Rollback using vector because of introducing a bug * 673 Cleanup commented out code --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature #2996 v12.0.0_beta6 (#2998) * Per #2996, add beta6 release notes. * Per #2996, slight change in order * Update docs/Users_Guide/release-notes.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Feature #2673 wchar argv (#2999) * Per #2673, switch to STL vector to attempt to squash a SonarQube bug. * Per #2673, remove m_strlen() entirely. * Per #2673, try again ci-skip-unit * Per #2673, switching to 2 more vectors * Update the release date for MET-12.0.0-beta6 * Feature #3000 point2grid_warning (#3001) * Per #3000, change 9 warning messages to Debug(2) log messages to minimize the number of warning messages Point2Grid writes. However, do retain the warnings about problems writing the grid mapping output file. * Per #3000 very minor whitespace diffs * Feature #2631 static_file_updates (#3002) * Per #2631, regenerate ndbc_stations.xml for MET-12.0.0 on October 22, 2024. * Per #2631, add MET_BASE anchor * Per #2631, add a description of some of the static data files that need to be periodically updated. Note that this is NOT an exhaustive list. * Per #2631, consistent wording * Per #2631, subset by update frequency. * Per #2631, consistent wording. Fix typos. * Per #2631, fix typo and URL * Per #2631, fix masked array error in the get_precision(...) function I encountered when using the utility to inspect the diffs for this PR. * Updating to add MET_PYTHON_EXE * Update compiler * Update install_met_env.casper to use oneAPI compilers * Based on feedback from @Ho-ChunHuang-NOAA, replace references to GOES-16 and GOES-17 with GOES-East and GOES-West in the MET User's Guide. * Feature #2780 grib tables (#3005) * Per #2780, grib2_bom.txt only has minor change replacing 'Prelim:' with 'Prelim' * Per #2780, grib2_tigge.txt add new table after eliminating whitespace from units strings * Per #2780, grib2_all.txt increase the number of entries from 1019 to 1570. * Per #2780, grib2_kma.txt add new grib2 table file. * Per #2780, carefully merge the existing 175 MRMS GRIB2 table entries with updates from the 109 defined in the wgrib2 table. * Per #2780, grib2_ecmwf.txt add new ECMWF GRIB2 table. * Per #2780, grib2_dwd.txt add new DWD GRIB2 table. * Per #2870, correct units out of order * Per #2870, grib2_gpp_af.txt elimiate embedded whitespace from units string to avoid problems writing them to FCST_UNITS and OBS_UNITS output columns * Per #2780, grib2_ecmwf.txt eliminate whitespace from units string * Per #2780, add/update GRIB2 tables * Per #2870, remove duplicate entries I'd added. * Per #2870, grib2_all.txt update with more entries from this version of wgrib2: hera:/home/Wesley.Ebisuzaki/grib2/wgrib2/gribtables/ncep/gribtable.dat * Per #2780, grib2_mrms.txt add entries for FRANA as requested by @MarcelCaron-NOAA. * Per #2780, add new grib1_jra_55_200.txt file that did not previously exist for MET. * Per #2780, remove embedded whitespace from GRIB1 DWD tables. * Per #2780, fix all the units by removing embedded whitespace and making them more human-readable by replacing ** with ^ to indicate exponentiation. Also correct bad units for radiation flux in AF tables by listing them as W/m^2. * Per #2780 eliminate trailing whitespace * Per #2870, fix typo * Per #2870, fix typos * Per #2870, fix typo * Per #2780, grib1_ncep_129_7.txt fix units by replacing 'µg' with 'mcg' to represent micrograms. Also replace 'mm6' with 'mm^6' by double-checking against the GRIB1 docs. * Per #2780, switch to defining particulate matter units as '10^-6g/m^3' for consistent usage across NCEP GRIB tables. * Per #2780, update GRIB2 units to show exponentiation correctly by comparing against documentation at https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc * Per #2780, more tweaks for consistent and corrected units * Per #2780, update Makefiles for new grib1_jra55_200.txt file * Per #2870, more tweaks for consistent formatting of units to eliminate the use of parantheses in the denominator. For example, replace 'kg/(m*s)' with 'kg/m/s'. * Feature #2729 tc_diag (#3011) * Per #2729, get rid of extra blank line in log message about manually deleting temp files. * Per #2729, fix log message describing the range/azimuth grid. * Per #2729, Update TC-Diag Python driver code to convert rotation angles from degree clockwise from due East to radians counter-clockwise from due East. * Per #2729, complete the TC-Diag documentation. * Per #2729, more tweaks * Per #2729, clarify that some diagnostics are directly extracted from the ATCF track file rather than being computed from the cylindrical coordinates grid. * Update docs/Users_Guide/tc-diag.rst Co-authored-by: KathrynNewman <knewman@ucar.edu> * Update docs/Users_Guide/tc-diag.rst Co-authored-by: KathrynNewman <knewman@ucar.edu> * Update docs/Users_Guide/tc-diag.rst Co-authored-by: KathrynNewman <knewman@ucar.edu> * Update docs/Users_Guide/tc-diag.rst Co-authored-by: KathrynNewman <knewman@ucar.edu> * Update docs/Users_Guide/tc-diag.rst Co-authored-by: KathrynNewman <knewman@ucar.edu> --------- Co-authored-by: KathrynNewman <knewman@ucar.edu> * Feature #2750 mtd_docs (#3014) * Per #2750, user-provided should be hyphenated. * Per #2750, add units to MTD output tables and specify order of forecast minus observation differences, where applicable. Also update some wording for consistency. * Unrelated to #2750, but prior to the v12.0.0 release, create a 12.0.0 flowchart, save as PNG, and use it to update the overview figure. * Per #2750, put X_DOT and Y_DOT back on separate lines since RTD interprets X_,Y_DOT as being a poorly defined link. * Feature 3009 net cdf docs (#3016) * Update data_io.rst Updated netCDF reading documentation * Update data_io.rst * Update data_io.rst * Update data_io.rst updated link * Update data_io.rst Fixed typo * Feature #3015 v12.0.0-rc1 (#3017) * Per #3015, update the release version number and add release notes. * Per #3015, add one upgrade instruction * update logic to be able to trigger METplus testing workflow for main_vX.Y and develop branches * fix typo in workflow * add quotes around value sent to trigger METplus workflow and remove unnecessary step * use custom GitHub Action to trigger METplus use case testing * Updating * Bugfix #3020 develop grid_stat_seeps (#3021) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Updated to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the develop branch after PR #3021 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Feature #2966 local_solar_time (#3008) * Per #2966, add new solar_time() function to the vx_solar library. * Per #2966, add support for new solar_time masking type. Also make log messages for consistent and eliminate the warning about -thresh not being specified becuase its fine to not specify a threshold. * Per #2966, add a units attribute to the output NetCDF mask variable. * Per #2966, modify solar azimuth and altitude strings to make the log messages align well. * Per #2966, add gen_vx_mask unit test to demonstrate the solar_time masking type. * Per #2966, add documentation about the -solar_time option * Per #2966, reduce SonarQube code smells in gen_vx_mask * Per #2966, reduce SonarQube findings * Per #2966, support multiple mask types with the same mask field being supported in a single run. Still need to update the user's guide. * Per #2966, add UTC * Per #2966, update gen_vx_mask docs about supporting multiple -type options in a single run * Per #2966, update logic to fix using data masking twice, add a unit test to demonstrate, and update the mask_type attribute to include the magic string for the gridded data used for data masking. * Per #2966, adjust the logic slightly to revert to existing behavior where we only write the timing information of the input data to the gen_vx_mask output when no threshold was applied. This should reduce the number of diffs flagged by PR #3008 * Per #2966, update details about the -union, -intersection, and -symdiff options in the usage statement and documentation as recommended by @CPKalb. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #3024 V12.1 (PR 1 of 2) (#3026) * Hotfix to the develop branch to add met_header_columns_V12.1.txt to data/table_files/Makefile.in * Adding In Memoriam * Feature #3012 ugrid for 3D points (PointXYZ) (#3035) * #3012 Initial release * #3012 Added search_3d_kdtree.cc * #3012 Initial release * #3012 Added check_llh_to_ecef and test_llh_to_ecef * #3012Added search_3d_kdtree_api * #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km * #3012 Compare points_XYZ or points_latlon * #3012 Commented out latlonalt_to_xy & xy_to_latlonalt * #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km. Added llh_to_ecef APIs * Per #3012, update the .gitignore settings to ignore the newly created test executables in the test_util/libcode/vx_grid directory. * Per #3012, changes to the vx_grid library to address SonarQube code smells in New Code and hopefully reduce the Overall number of code smells. * Per #3012, couple more minor SonarQube tweaks --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #3038 nc_dim_log (#3039) * Per #3038, add debug and warning log messages to tell the user which NetCDF dimension indices are being used. * Per #3038, fix a few SonarQube code smells to reduce the overall number of them for the #3039 PR. * Feature 2709 mvmode multithreshradii (#3034) * Source code changes for multiple radii/thresholds in multivariate mode * Slight cleanup * added multiple thresh/radii functionality for the mvmode superobjects only situation * mvmode log message improvements * Mods to the mode users guide for multiple convolution thresholds and radii in multivariate mode * Added v12.1 * Minor bugfix and tweak * Per #2709, updates to fix SonarQube code smells. * Per #2709, fix compilation problem I introduced with SonarQube fixes. * Per #2709, eliminate SonarQube bug that was flagged * Per #2709, fix typo in docs --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #3032 develop docs_data_types (#3041) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Feature #3033 MTD header columns (#3043) * Doc-only change to rename the 'Code support' section as 'User support' based on 1/9/2025 decision at the METplus-Analysis Tools project meeting. Also update MET to link to the METplus User's Guide. * Fix typo, missing one * to make SciPy bold in appendixF.rst * Update grid-stat.rst to correctly list the GRAD DX and DY columns as Integers (not Doubles). * Feature #3024 and #3030 Series-Analysis GRAD (#3036) * Per #3024, roll the version number from 12.0 to 12.1 prior to adding new columns to the existing GRAD line type. * Per #3024, return bad data for the square root of a negative number * Per #3024, update Grid-Stat to write 4 new columns to the GRAD line type. * Per #3024, tweak laplace MSE computation * Per #3024, add appendixC.rst equations for the new statistics * Per #3024, escape embedded underscores in equations only for latex formatting, not html * Per #3024, add reference to paper and summary of the new gradient stats * Per #3030, update docs by moving the gradient config entry from Grid-Stat to the common area and then referencing it in both Grid-Stat and Series-Analysis. * Per #3030, update Series-Analysis config file by adding the gradient dictionary and an entry for output_stats.gradient. Update the conf_info source code to parse them. Still need to update OTHER Series-Analysis config files and also update the logic in series_analysis.cc to compute GRAD statistics. * Per #3030, add descriptions of GRAD statistics for inclusion in the long_name attribute of the Series-Analysis output files. * Per #3030, add the GRADInfo::get_stat() member function to be called by Series-Analysis * Per #3030, fix parsing logic for the gradient dictionary. * Per #3030, add gradient entries to existing Series-Analysis config files * Per #3030, add GRADInfo::set_stat() member function * Per #3030, enhance Series-Analysis to aggregate the GRAD line type incrementally across multiple runs. However, this can only be done when requesting that 'ALL' GRAD columns be written. * Per #3030, reduce the volume of SonarQube code smells. * Updated syntax * Updated syntax * Updated syntax --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3053) * Feature #1019 USCRN (#3049) * Per #1019, initial setup for supporting -format uscrn. It compiles but I still need to make it work for the variety of USCRN inputs. * Per #1019, add NumArray constructor using a vector of doubles. * Per #1019, saving progress after handling monthly, daily, and hourly input files. Need to complete support for other format types and handle the unit strings * Per #1019, consistent spacing. * Per #1019, tweak log messages so that the file being read is logged before it's actually read so that an error in parsing the data will indicate which file caused it. * Per #1019, update DataLine and LineDataFile classes to support parsing .csv files. Get rid of the unneeded Offsets vector. Add AllowEmptyColumns option to the DataLine class so that multiple delimiters in a row will be treated as separate columns. Since the default delim is whitespace, it makes sense that you'd want to parse multiple delims in a group. But for .csv files, each comma indicates a new column. * Per #1019, update USCRN handler code to support all 7 input variants, including .csv files. This required updates to the DataLine and LineDataFile classes to parse the .csv data properly. Still need to enhance ascii2nc to write units * Per #1019, add units to the Observation class. * Per #1019, add units string to the SummaryObs and SummaryKey classes. * Per #1019, update USCRN format to write units. Consider defining units for all the other ascii file types as well. * Per #1019, add StringArray::all_empty() member function to check for a list of all empty strings. This is used in ascii2nc to determine if observation units and descriptions should be written. * Per #1019, update library code to handle the independent writing of point observation descriptions. Previously, if units were present then descriptions (usually empty ones) were added. Now, units and descriptions and handled independently. * Per #1019, add descriptions for all USCRN observations, pulled from the USCRN website. * Per #1019, need to subtract 1900 from the year and 1 from the month to make this work. Seems like we should ADD these numbers where needed rather than subtracting them everywhere else! * Per #1019, get rid of USCRNHandler::_readHeaderInfo() since it compilicated the logic for ignoring the first line from csv files. * Per #1019, rather than always skipping the first line of USCRN csv files, just skip any lines where the station ID begins with 'WBAN'. That'll handle files being concatenated together and is simpler logic. * Per #1019, add an ascii2nc unit test for USCRN point observations. …
georgemccabe
pushed a commit
to dtcenter/METplus
that referenced
this pull request
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a rather extensive set of changes to more broadly support wind rotation and derivation of gridded wind data across multiple input file types. Please note the following development details:
set_job_controls.shreverts back to METbaseimage 3.4 for now since 3.5 causes an unrelated segfault in TC-Diag. This needs to be fixed separately.ConfigConstantsdefaults show common GRIB and WRF-NetCDF variable names. Please advise if these should be updated.unit_plot_data_plane.xml, I addedis_grid_relative=false;in 2 spots to suppress wind rotation to preserve existing output and demonstrate that settingis_grid_relativeactually works.unit_winds.xmlcall pcp_combine as a "pass-through" to demonstrate the rotation and derivations of winds from both U/V and wind speed/direction inputs.WindMetaDatastruct stores the metadata for identifying winds info in one spot. It is an element of VarInfo and can be specified separately for EACH FIELD read, if needed.vx_data2dbase class library.read_wind_data()function, when going U->V or V->U, it FIRST tries swapping the U's and V's to look for a match. If the swap changes the field name, we try that first. If not, we proceed to the search strings in the*_wind_field_nameoption.set_name()was NOT sufficient. Instead we need the fullset_dict()option to include all the misc config options.table_lookup.h/.ccshould be noted. Previously, MET logged when a variable named (e.g.TMP) appeared more than once in the GRIB tables. Of course that happens all the time! That's now changed to only print info about the name NEVER appearing in the GRIB tables.get_field()reach each config string independently rather than letting one string settings affect the subsequent ones.Expected Differences
If yes, please describe:
Adds the following config options in
ConfigConstants:If yes, please describe:
Pull Request Testing
Describe testing already performed for these changes:
Extensive manual testing during development and added 3 new unit tests in
unit_winds.xmlto illustrate the functionality for GRIB1, GRIB2, and WRF-NetCDF input files.Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
unit_winds.xmlat verbosity level 3 and carefully review the log messages about the derivation and rotation of winds.seneca:/d1/projects/MET/MET_pull_requests/met-13.0.0/rc1/MET-feature_1518_winds.Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
Do these changes include sufficient testing updates? [Yes]
unit_winds.xmldemonstrate with 3 input files. Please advise if testing for additional inputs is warranted.Will this PR result in changes to the MET test suite? [Yes]
If yes, describe the new output and/or changes to the existing output:
unit_winds.xmlare added.grib_tables/um_raw/PS.557WW_SC.U_DI.F_GP.GALWEM-GD_GR.C17KM_AR.GLOBAL_DD.20160424_CY.00_FH.000_DF.psgrid_stat/grid_stat_WRF_pres_120000L_20190831_120000V_pairs.ncandgrid_stat/grid_stat_WRF_pres_120000L_20190831_120000V.statU_PLdata (e.g. U-wind at pressure levels) was NOT being rotated from grid-to-earth relative, and now it is.Will this PR result in changes to existing METplus Use Cases? [Maybe]
If yes, create a new Update Truth METplus issue to describe them.
Do these changes introduce new SonarQube findings? [Yes]
If yes, please describe:
Please complete this pull request review by [Friday, July 3rd].
I scheduled a meeting to discuss for Tuesday, June 30th.
Pull Request Checklist
See the METplus Workflow for details.
Select: Reviewer(s) and Development issue
Select: Milestone as the version that will include these changes
Select: METplus-X.Y Support project for bugfix releases or MET-X.Y Development project for the next coordinated release