Exception when using mth5 version 0.6.0, mt-metadata 1.0.0

[jameswilburlewis]

I have a test that works fine under mth5 0.5.0 or 0.5.1, but fails under mth5 version 0.6.0 with this traceback:

.ss
======================================================================
ERROR: test02_load_fdsn_basic (__main__.TestMTH5LoadFDSN.test02_load_fdsn_basic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/pyspedas/mth5/tests/test_load_fdsn.py", line 118, in test02_load_fdsn_basic
    load_fdsn(network="4P", station="REU49", trange=[date_start, date_end])
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/pyspedas/mth5/load_fdsn.py", line 176, in load_fdsn
    mth5_path = fdsn_object.make_mth5_from_fdsn_client(request_df, interact=False, path=mth5dir)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/mth5/clients/fdsn.py", line 458, in make_mth5_from_fdsn_client
    return self.make_mth5_from_inventory_and_streams(inv, streams, save_path=path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/mth5/clients/fdsn.py", line 516, in make_mth5_from_inventory_and_streams
    m.from_experiment(experiment)
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/mth5/mth5.py", line 1119, in from_experiment
    sg = self.add_survey(survey.id, survey_metadata=survey)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/mth5/mth5.py", line 1238, in add_survey
    return self.surveys_group.add_survey(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pyspedas/.venv/lib/python3.12/site-packages/mth5/groups/survey.py", line 153, in add_survey
    raise ValueError("survey name is None, do not know what to name it")
ValueError: survey name is None, do not know what to name it

The exception is happening at this call to fdsn_object.make_mth5_from_fdsn_client:

        fdsn_object = FDSN(mth5_version='0.2.0', client="IRIS")
        try:
            mth5_path = fdsn_object.make_mth5_from_fdsn_client(request_df, interact=False, path=mth5dir)

The values in request_df at the point of the failure are:

network: '4P'
station: 'REU49'
location: '--'
channel: 'MFZ,MFN,MFE,LFZ,LFN,LFE,VFZ,VFN,VFE,UFZ,UFN,UFE'
start: '2015-06-22T01:45:00'
end: '2015-06-22T02:20:00'

I get this console output:

2026-01-26T18:06:43.200788-0800 | INFO | mth5.mth5 | _initialize_file | line: 773 | Initialized MTH5 0.2.0 file mth5/4P_REU49.h5 in mode w
2026-01-26T18:06:43.207135-0800 | INFO | mth5.mth5 | close_mth5 | line: 896 | Flushing and closing mth5/4P_REU49.h5
26-Jan-26 18:06:43: Cannot initialize mth5 object:
survey name is None, do not know what to name it
26-Jan-26 18:06:43: Deleting mth5 temporary h5 file 4P_REU49.h5

Has something in the API changed between 0.5.1 and 0.6.0? Is there something I can do to fix this in my code?

[kujaku11]

@jameswilburlewis There has been some changes in 0.6.0, the main change was adapting to the mt_metadata update to 1.0.0, which was a major update to use Pydantic for all metadata objects, but the functionality of mth5 should not have changed much.

From the error you posted, the test:

 def test04_load_fdsn_samedates(self):
        """
              If dates are the same, tplot variable should not be created.
              load_fdsn should return none
        """
        # The server response has changed to '422: Unprocessable Entity' for this request
        # because of the same start/end times.  So we need to catch the exception now. 
        ### REPLY: Yes for what ever reason a 422 is returned when there is no data to return
        ### so your catch is correct.
        from obspy.clients.fdsn.header import FDSNException

        date_start = '2015-06-22T01:45:00'
        date_end = date_start
        tvar = None

        try:
            tvar = load_fdsn(network="4P", station="REU49", trange=[date_start, date_end])
        except FDSNException as e:
            pass

        self.assertIsNone(tvar)

But maybe mth5 should return a error or do a try/except and return a message saying no data was returned. Would you prefer an error message or a logger warning message and None returned?

I'm looking into the what is causing the exception for test02.

[jameswilburlewis]

Thanks for responding -- I've already resolved the issue with test04_load_fsdn_samedates, by checking the start/end times before attempting the query. So if end_time <= start_time, we generate an ordinary error message instead of letting the mth5 code query the server and raise an exception. Internal to MTH5, raising an exception for that case seems like a reasonable way to handle it, and it was easy for us to see what the problem was, so I don't think that needs to be changed on your end.

But we're still very interested in understanding the failure in test02. For now. we've added upper-version pins for the mth5/mt-metadata to ensure that we're using mth-0.5.1 or below.

[kujaku11]

@jameswilburlewis Thanks for the quick response and finding this bug. I found the issue, it is in mt_metadata.

When the obspy inventory object is being translated to an mt_metadata.timeseries.Survey object the inventory.network[0].alternate_code was None which is what the Survey.id is supposed to translate to. There was logic to catch a None value and set it to the network.code. However, in the translation to Pydantic a None becomes an empty string, so when Survey.id = None was validated to Survey.id = "". So the id was kept at "" because the logic was only checking for a None.

I have updated this in the patches branch of mt_metadata and will release this weekend after some clean up and additional bug fixes.

If you want a simple fix change line 145 in mt_metadata.timeseries.stationxml.xml_network_mt_survey to if mt_survey.id in [None, ""]:

[kujaku11]

@jameswilburlewis This should be closed with updated mt-metadata >= 1.0.2. Please confirm this fixed the issue.

[jameswilburlewis]

tt's not failing inside the mth5 library any more, but it seems that the interface to station_data.get_run(run_name=run_name) has changed to return a mt_metadata.common.mttime.MTime object rather than a string, and so the test fails.

This is under Python 3.10, MacOS Sonoma 14.8.3, with the following mth5 0.6.4, obspy 1.4.2, mt-metadata 1.0.4:

Error
Traceback (most recent call last):
  File "/Users/jwl/PycharmProjects/pyspedas_dev_310/pyspedas/mth5/tests/test_load_fdsn.py", line 119, in test02_load_fdsn_basic
    load_fdsn(network="4P", station="REU49", trange=[date_start, date_end])
  File "/Users/jwl/PycharmProjects/pyspedas_dev_310/pyspedas/mth5/load_fdsn.py", line 240, in load_fdsn
    run_start = datetime.fromisoformat(run_data.metadata.time_period.start).replace(tzinfo=None)
TypeError: fromisoformat: argument must be str

The surrounding context is:

            for run_name in station_data.metadata.run_list:
                # Get the data
                run_data = station_data.get_run(run_name=run_name)

                # Check run time coverage
                run_start = datetime.fromisoformat(run_data.metadata.time_period.start).replace(tzinfo=None)
                run_end = datetime.fromisoformat(run_data.metadata.time_period.end).replace(tzinfo=None)

                # Skip processing if run is outside requested time range
                if run_start > request_end or run_end < request_start or request_end <= request_start:
                    continue

When I set a breakpoint there, at the assignment to run_start, run_data.metadata.time_period.start is indeed not a str:

type(run_data.metadata.time_period.start)
Out[2]: mt_metadata.common.mttime.MTime

After reverting to mth5 0.5.1, obspy 1.4.2, mt-metadata 0,4,0, at the same breakpoint that value was definitely a string before:

type(run_data.metadata.time_period.start)
Out[3]: str

I see that with the latest package versions, calling str(run_data.metadata.time_period.start) does return an ISO time string, so I can probably work with that. But it would be good to know if that's the intended API behavior I can rely on going forward, or a bug.