dtcenter · mkavulich · Jul 9, 2026 · May 27, 2026 · May 27, 2026 · May 28, 2026
@@ -21,7 +21,7 @@ If an issue already exists, please use that issue to add any additional informat
 <!-- Tell us what happens instead of the expected behavior. -->
 
 ## Machines affected
-<!--- Please provide any relevant information about your setup, including machine/compiler combination. -->
+<!--- Please provide any relevant information about your setup, including machine and MET/METplus version if relevant. -->
 <!-- Reference other issues or PRs in other repositories that this issue is related to, and how they are related. -->
 
 ## Steps To Reproduce

@@ -22,13 +22,9 @@ Please search on the [Issue tracker](https://github.com/ufs-community/ufs-srweat
 ## Solution
 <!-- Add a clear and concise description of the proposed solution. -->
 
-## Requirements**
-<!-- What does the new code need to accomplish? Does it require an update 
-to a version of software (e.g. modules of NCEPLibs, NetCDF, etc.), components 
-(e.g. UFS-Weather-Model), or system tools (e.g. python3) -->
-
-## Acceptance Criteria (Definition of Done)
-<!-- What does it mean for this feature to be finished? -->
+## Requirements
+<!-- What does the new code need to accomplish? Does it require a specific/newer
+version of MET/METplus? -->
 
 ## Dependencies (optional)
 <!-- Directly reference any issues or PRs in this or other repositories that this 

@@ -1,15 +1,10 @@
-
-- Update develop to head at ufs-community
-
 - Use this template to give a detailed message describing the change you want to make to the code.
 
 - You may delete any sections labeled "optional" and any instructions within <!-- these sections -->.
 
 - If you are unclear on what should be written here, see https://github.com/wrf-model/WRF/wiki/Making-a-good-pull-request-message for some guidance and review the Code Contributor's Guide at https://github.com/ufs-community/ufs-srweather-app/wiki/Code-Manager's-Guide. 
 
-- Code reviewers will assess the PR based on the criteria laid out in the Code Reviewer's Guide (https://github.com/ufs-community/ufs-srweather-app/wiki/Code-Manager's-Guide). 
-
-- The title of this pull request should be a brief summary (ideally less than 100 characters) of the changes included in this PR. Please also include the branch to which this PR is being issued (e.g., "[develop]: Updated UFS_UTILS hash").
+- The title of this pull request should be a brief summary (ideally less than 100 characters) of the changes included in this PR.
 
 - Use the "Preview" tab to see what your PR will look like when you hit "Create pull request"
 
@@ -24,62 +19,40 @@
 - [ ] Bug fix (non-breaking change which fixes an issue)
 - [ ] New feature (non-breaking change which adds functionality)
 - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
-- [ ] This change requires a documentation update
+- [ ] Text only (documentation, README files, code comments, etc.)
 
 ## TESTS CONDUCTED: 
-<!-- Explicitly state what tests were run on these changes, or if any are still pending (for README or other text-only changes, just put "None required"). Make note of the compilers used, the platform/machine, and other relevant details as necessary. For more complicated changes, or those resulting in scientific changes, please be explicit! -->
+<!-- Explicitly state what tests were run on these changes, or if any are still pending (for README or other text-only changes, just put "None required"). Make note of the platform/machine and other relevant details as necessary. For more complicated changes, or those resulting in scientific changes, please be explicit! -->
 <!-- Add an X to check off a box. -->
 
-- [ ] derecho.intel
-- [ ] gaeac6.intel
-- [ ] hera.gnu
-- [ ] hera.intel
-- [ ] hercules.intel
-- [ ] orion.intel
-- [ ] ursa.gnu
-- [ ] ursa.intel
-- [ ] NOAA Cloud (indicate which platform)
-- [ ] Jenkins
-- [ ] fundamental test suite
-- [ ] comprehensive tests (specify *which* if a subset was used)
+- [ ] derecho
+- [ ] hera
+- [ ] hercules
+- [ ] orion
+- [ ] ursa
+- [ ] other (describe)
 
 ## DEPENDENCIES:
-<!-- Add any links to external PRs (e.g. regional_workflow and/or UFS PRs). For example:
-- ufs-community/regional_workflow/pull/<pr_number>
-- ufs-community/UFS_UTILS/pull/<pr_number>
-- ufs-community/ufs-weather-model/pull/<pr_number> -->
+<!-- Add any links to PRs (internal or external) that must be merged before this one. For example:
+- #23
+- https://github.com/dtcenter/METplus/pull/3301
 
 ## DOCUMENTATION:
 <!-- If this PR is contributing new capabilities that need to be documented, please also include updates to the RST files (docs/UsersGuide/source) as supporting material. -->
 
 ## ISSUE: 
 <!-- If this PR is resolving or referencing one or more issues, in this repository or elsewhere, list them here (Remember, issues must always be created before starting work on a PR branch!). For example, "Fixes issue mentioned in #123" or "Related to bug in https://github.com/ufs-community/other_repository/pull/63" -->
 
-## CHECKLIST
-<!-- Add an X to check off a box. -->
-- [ ] My code follows the style guidelines in the Contributor's Guide
-- [ ] I have performed a self-review of my own code using the Code Reviewer's Guide
-- [ ] I have commented my code, particularly in hard-to-understand areas
-- [ ] My changes need updates to the documentation. I have made corresponding changes to the documentation
-- [ ] My changes do not require updates to the documentation (explain).
-- [ ] My changes generate no new warnings
-- [ ] New and existing tests pass with my changes
-- [ ] Any dependent changes have been merged and published
-
 ## LABELS (optional): 
 <!-- If you do not have permissions to add labels to your own PR, request that labels be added here. 
 Add an X to check off a box. Delete any unnecessary labels. -->
 A Code Manager needs to add the following labels to this PR: 
-- [ ] Work In Progress
+- [ ] Work in progress
 - [ ] bug
-- [ ] enhancement
 - [ ] documentation
-- [ ] release
-- [ ] high priority
-- [ ] run_ci
-- [ ] run_we2e_fundamental_tests
-- [ ] run_we2e_comprehensive_tests
+- [ ] enhancement
 - [ ] help wanted
+- [ ] high priority
 
 ## CONTRIBUTORS (optional): 
 <!-- If others have contributed to this work aside from the PR author, list them here -->

@@ -35,7 +35,7 @@ jobs:
           micromamba activate vx_workflow
           export PYTHONPATH=$(pwd)/ush
           pylint --version
-          pylint --ignore-imports=yes tests/test_python/
+          pylint --ignore-imports=yes tests/test_python/test_*.py
           pylint ush/generate_wflow.py
           pylint ush/setup.py
           pylint scripts/*.py
@@ -49,9 +49,9 @@ jobs:
 
           python -m unittest discover -s tests/test_python -p "test_*.py"
 
-      - name: Run python functional tests
-        run: |
-          micromamba activate vx_workflow
-          export CI=true
-          export PYTHONPATH=${PWD}/ush
-          python3 -m unittest tests.test_python.test_retrieve_data
+#      - name: Run python functional tests
+#        run: |
+#          micromamba activate vx_workflow
+#          export CI=true
+#          export PYTHONPATH=${PWD}/ush
+#          python3 -m unittest tests.test_python.test_retrieve_data
@@ -145,7 +145,6 @@ The |topdir| structure follows the standards laid out in the :term:`NCEP` Centra
        ├── get_metplus_tool_name.sh
        ├── get_obs.py
        ├── launch_vx_wflow.sh
-       ├── load_modules_wflow.sh*
        ├── machine/
        ├── python_utils/
        ├── retrieve_data.py*

@@ -467,4 +467,11 @@ GOESADP:
     file_names:
       obs:
         - "OR_ABI-L2-ADPF-M[3-6]_G16_s{fyyyy}{fjjj}{fhh}*.nc"
+BDECK:
+  ftp:
+    protocol: wget
+    url: "https://ftp.nhc.noaa.gov/atcf/archive/{yyyy}/"
+    file_names:
+      obs:
+        - "bal{cyclone}{yyyy}.dat.gz"
 
@@ -41,10 +41,10 @@ INPUT_BASE = {{fcst_input_dir}}
 OUTPUT_BASE = {{output_dir}}
 
 TC_PAIRS_ADECK_INPUT_DIR = {INPUT_BASE}
-TC_PAIRS_BDECK_INPUT_DIR = {{best_track_dir}}
+TC_PAIRS_BDECK_INPUT_DIR = {{output_dir}}
 
 TC_PAIRS_ADECK_TEMPLATE = {{fcst_track_file}}
-TC_PAIRS_BDECK_TEMPLATE = b{basin}{cyclone}{init?fmt=%Y}.dat
+TC_PAIRS_BDECK_TEMPLATE = {{best_track_file}}
 TC_PAIRS_READ_ALL_FILES = no
 
 TC_PAIRS_OUTPUT_DIR = {OUTPUT_BASE}

@@ -35,7 +35,10 @@ task_tcpairs:
       cyclestr:
         value: '@Y@m@d@H'
   memory: '{{  tcpairs.execution.memory if user.MACHINE not in ["NOAACLOUD"] }}'
+  native: '{% if platform.get("SCHED_NATIVE_CMD_HPSS") %}{{ platform.SCHED_NATIVE_CMD_HPSS }}{% else %}{{ platform.SCHED_NATIVE_CMD}}{% endif %}'
   nodes: '{{ verification_resources.execution.nodes }}:ppn={{ tcpairs.execution.tasks_per_node }}'
+  partition: '{{ "&PARTITION_HPSS;" if platform.get("PARTITION_HPSS") }}'
+  queue: "&QUEUE_HPSS;"
   walltime: "{{ tcpairs.execution.walltime }}"
 
 task_tcstat:

@@ -870,7 +870,7 @@ def get_obs(config, obtype, yyyymmdd_task):
                         if obtype == 'CCPA':
                             fn_raw = f'ccpa.t{hr:02d}z.{accum_obs_formatted}h.hrap.conus.gb2'
                         elif obtype == 'NOHRSC':
-                            fn_raw = 'sfav2_CONUS_{accum_obs_formatted}h_{yyyymmddhh_str}_grid184.grb2' # pylint: disable=line-too-long
+                            fn_raw = f'sfav2_CONUS_{accum_obs_formatted}h_{yyyymmddhh_str}_grid184.grb2' # pylint: disable=line-too-long
                         elif obtype == 'MRMS':
                             #MRMS files are retrieved from HPSS archives as gzip; need to unzip
                             with gzip.open(valid_file_name, 'rb') as f_in:

@@ -295,10 +295,12 @@ mkdir -p "${OUTPUT_DIR}"
 #
 #-----------------------------------------------------------------------
 #
-if [ ! -d "${OBS_DIR}" ]; then
-  print_err_msg_exit "\
-OBS_DIR does not exist or is not a directory:
-  OBS_DIR = \"${OBS_DIR}\""
+if [ "${FCST_OR_OBS}" = "OBS" ]; then
+  if [ ! -d "${OBS_DIR}" ]; then
+    print_err_msg_exit "\
+  OBS_DIR does not exist or is not a directory:
+    OBS_DIR = \"${OBS_DIR}\""
+  fi
 fi
 #
 #-----------------------------------------------------------------------

@@ -6,15 +6,18 @@
 The script is intended to be called from jobs/TCPAIRS.sh.
 """
 import argparse
+import gzip
 import logging
 import os
+import shutil
 
 from multiprocessing import Pool
 from pathlib import Path
 
 import uwtools.api.config as uwconfig
 
-from python_utils import setup_logging, render_metplus_confs, run_metplus
+from python_utils import eval_metplus_timestr_tmpl, render_metplus_confs, run_metplus, setup_logging
+import retrieve_data
 
 def tcpairs(config_file, cdate):
     """
@@ -53,7 +56,42 @@ def tcpairs(config_file, cdate):
     os.makedirs(output_dir, exist_ok=True)
 
     conf_files=[]
-    for storm_id in tccfg['STORM_IDS']:
+    for storm in tccfg['STORM_IDS']:
+        storm_id=f"{storm:02}"
+        # Ensure best track file is present, and if not, retrieve it:
+        best_track_template=tccfg["BEST_TRACK_FILE"]
+        # Need to substitute keywords manually to check file existence
+        best_track_fp = eval_metplus_timestr_tmpl(best_track_template, cdate,
+                                                  cyclone=storm_id, basin=tccfg["BASIN"])
+        if os.path.exists(best_track_fp):
+            best_track_file = os.path.basename(best_track_fp)
+            shutil.copy(best_track_fp, output_dir)
+        else:
+            lgr.info(f"{best_track_fp=} does not exist on disk, attempting to download...")
+            dataargs = ['--debug', \
+                    '--file_set', 'obs', \
+                    '--config', os.path.join(cfg['user']['PARMdir'], 'data_locations.yml'), \
+                    '--cycle_date', cdate, \
+                    '--cyclone', storm_id, \
+                    '--data_stores', "ftp", \
+                    '--data_type', "BDECK", \
+                    '--output_path', str(output_dir), \
+                    '--summary_file', 'retrieve_data.log']
+            lgr.debug(f'{dataargs=}')
+            retrieve_data.main(dataargs)
+            # NEED TO RETRIEVE THIS VALUE RETURNING FROM retrieve_data AFTER REWRITE
+            bt_zipfile = eval_metplus_timestr_tmpl('bal{cyclone}{init?fmt=%Y}.dat.gz', cdate,
+                                                        cyclone=storm_id, basin=tccfg["BASIN"])
+            best_track_zipfp = Path(output_dir,bt_zipfile)
+            best_track_file = bt_zipfile.rsplit( ".", 1 )[ 0 ]
+            best_track_fp_out = best_track_zipfp.with_suffix("")
+            lgr.debug(f"Extracting retrieved zipfile {best_track_zipfp} to {best_track_fp_out}")
+            # Extract the zip file into output_dir
+            with gzip.open(best_track_zipfp, 'rb') as file_in:
+                with open(best_track_fp_out, 'wb') as file_out:
+                    shutil.copyfileobj(file_in, file_out)
+                    lgr.debug(f'{best_track_fp_out} file created')
+
         # Set the names of the template METplus configuration file, the resulting rendered conf
         # file, and the METplus log file
         metplus_config_tmpl_fn="TCPAIRS.conf"
@@ -78,7 +116,8 @@ def tcpairs(config_file, cdate):
                    'basin': tccfg['BASIN'],
                    'storm_id': storm_id,
                    'fcst_track_file': tccfg['ADECK_TEMPLATE'],
-                   'best_track_dir': cfg["platform"]["BEST_TRACK"]
+                   'best_track_file': best_track_file,
+                   'best_track_dir': output_dir
                    }
 
         numprocs=1
@@ -91,9 +130,16 @@ def tcpairs(config_file, cdate):
     for config_fn in conf_files:
         args.append( (os.path.join(cfg['user']['METPLUS_CONF'], "common.conf"),config_fn) )
     # Call run_metplus function for as many processors as specified
-        lgr.debug(f"{args=}")
-    with Pool(processes=numprocs) as pool:
-        pool.starmap(run_metplus,args)
+    lgr.debug(f"{args=}")
+    try:
+        with Pool(processes=numprocs) as pool:
+            pool.starmap(run_metplus,args)
+    except Exception:
+        lgr.error(
+            f"METplus {metplus_tool_camel_case} failed. "
+            f"Check the METplus log file(s) for details:"
+        )
+        raise SystemExit(1) from None
 
     lgr.info(f"{metplus_tool_camel_case} completed successfully.")