Release pyDARN version 4.3

[carleyjmartin]

Scope

Version 4.3 - Minor Release!

This minor release includes:

• Updated SciPy restriction and changes associated
• NEW: True velocity in map plots
• NEW: FITACF data detrending algorithm
• NEW: Snap to a FOV of a given radar option
• NEW: Option to plot only groundscatter or only ionospheric scatter in fan and RT plots
• Bug Fix: Bug fix in plot_center for FOV that didn't appropriately use the magnetic longitude
• Bug Fix: Updating HDW files on Windows fixed

This minor release has been requested by users, please can we test within the next 2 weeks.
Deadline: 3rd June for testing, 5th June for release.

Approval

Number of approvals: 3

Test

Note testers: please indicate what version of matplotlib and python you are using

I can offer test code, but it would be much better if you could formulate your own. You can find test code on the individual PR's recently in develop.
Please test at minimum:

• Map plots, with True velocity options
• Data detrending with FITACF data and plotting on range time/fan plots
• Lots of Fan plots and various options

Documentation

Docs link

Please read through the tutorials, you can use the test code in there to test which would be great as that also edits the docs too.

[Wtristen]

Yes, I'm at Virginia Tech still :) Here is my ORCiD: https://orcid.org/0009-0007-0616-5796

[Wtristen]

Testing Feedback

Branch: develop
Python Version: 3.13.5
Matplotlib Version: 3.10.9
Setup: Fresh install w/ dedicated environment
Method:
Wrote a test script that performs:
1.) plotting w/ MapParams.TRUE_VELOCITY
2.) detrend_fitacf() w/ type 'mean' and 'savgol', plotted both with RTP and FAN
Results:
1.) Successful hdw download, read
2.) Map plot successful
3.) De-trending & plotting performs as expected in all cases for a full-day fitacf file, but an error occurs with savgol on the test file 'test.fitacf.bz2'. Suspected to be an edge case caused by:

• Filter window exceeding data length: The filter window size (window_len), calculated in records from the user-specified window_length (in seconds), can be larger than the number of available time-series points. This occurs when the input data file contains fewer scans than the calculated window. This triggers a ValueError from SciPy: window_length must be less than or equal to the size of x.
• Polynomial order exceeds filter window: The calculated window_len can be less than or equal to the polyorder (which defaults to 2). This violates the requirement that fitting a polynomial of order P requires more than P data points. This triggers a ValueError from SciPy: polyorder must be less than window_length.

[carleyjmartin]

@Wtristen

3.) De-trending & plotting performs as expected in all cases for a full-day fitacf file, but an error occurs with savgol on the test file 'test.fitacf.bz2'.

Thanks!

For the error in the detrending test - was the error you got indicative of the problem or should we make a warning to give the user more information about why it broke?

[Wtristen]

@carleyjmartin

3.) De-trending & plotting performs as expected in all cases for a full-day fitacf file, but an error occurs with savgol on the test file 'test.fitacf.bz2'.

For the error in the detrending test - was the error you got indicative of the problem or should we make a warning to give the user more information about why it broke?

These are two edge cases that only occur for shorter fitacfs like the test file and are unlikely with the standard 2-hour files. We could add a guard by attempting to truncate the window on our end and issue some warnings inside detrend.py and in the detrend_savgol method:

_valid = np.sum(~mask) #<--existing line 72
if len(y_interp) < window_len or _valid < window_len:
    warnings.warn(
        f"window_len={window_len} exceeds available data. Truncating window.",
        UserWarning
    )
    polyorder = kwargs.get('polyorder', 2)
    window_len = min(window_len, len(y_interp))
    if window_len % 2 == 0:
        window_len -= 1
    if window_len <= polyorder:
        warnings.warn(
            f"After truncation, window_len ({window_len}) <= polyorder ({polyorder}). "
            f"Reduce polyorder below window_len or use a longer file. Returning zeros.",
            UserWarning
        )
        return [None if mask[i] else 0.0 for i in range(len(timeseries))]