supp_url and paper_url contain expired pre-signed CDN links and Scholar redirect URLs

[larnsce]

Observed

In the 0.1.0 release:

• 252 of 924 washdev rows carry pre-signed Silverchair CDN links in their URL columns, of the form https://iwa.silverchair-cdn.com/...pdf?Expires=1706027022&Signature=.... The Expires timestamps are from January 2024, so every one of these links is dead for reusers of the dataset.
• 3 uncnewsletter rows carry Google Scholar alert redirect URLs (https://scholar.google.com/scholar_url?url=...&scisig=...) instead of the target article URL.

Side effect: the high-entropy Signature= and scisig= parameters trip secret scanners on downstream ingest pipelines, which is how this surfaced.

Expected

URL columns hold stable, canonical locations:

• for the Silverchair cases, the DOI URL or the article landing page rather than the time-limited CDN link the browser happened to resolve at collection time;
• for the Scholar-alert cases, the url= target decoded out of the redirect.

Both rewrites are mechanical (parse the target out of the existing values), so no re-collection is needed where a DOI or target URL is embedded in the current string.

[larnsce]

Fixed on dev (commit c355e1a). Both rewrites were mechanical, no re-collection needed.

What changed

• washdev supp_url: 343 expired Silverchair pre-signed CDN links are rewritten to https://doi.org/ URLs. The DOI is recovered from the link path (e.g. .../10.2166_washdev.2011.015/... becomes https://doi.org/10.2166/washdev.2011.015). Where a row had multiple expired tokens (separate supplement files sharing one signature), they collapse to the single article DOI URL. Verified against the 87 rows that already carried a doi: the path-derived DOI matched the row's doi in every case.
• uncnewsletter paper_url: the 3 Google Scholar alert redirects are decoded to their target URLs out of the url= parameter (e.g. an AJPH, an ACS, and a Wiley article).

The high-entropy Signature and scisig tokens that tripped secret scanners are gone. Zero silverchair-cdn, Signature=, scholar_url, or scisig strings remain in either dataset.

Note on the numbers

The original report counted 252 affected washdev rows in 0.1.0, all in the url column. After the 0.2.0 rescrape the expired links sit in supp_url instead, and there are now 343 of them (433 individual tokens across those rows); paper_url is clean. The fix targets supp_url accordingly.

How

Two reproducible helpers in data-raw/helpers.R, canonicalize_silverchair_url() and decode_scholar_redirect(), applied in data-raw/data_processing.R. Only washdev supp_url and uncnewsletter paper_url changed; row counts and every other column are identical, and ploswater is untouched. Package check passes with 0 errors, 0 warnings, 0 notes.

This also unblocks the Accessible dimension of the FAIR rubric (#19), which needs resolvable links.

[larnsce]

Closing: the expired links are fixed. The 343 washdev supp_url values that were pre-signed Silverchair CDN links are rewritten to stable https://doi.org/ URLs, and the 3 uncnewsletter paper_url Google Scholar redirects are decoded to their target URLs (commit c355e1a, merged to main in PR #29). The rewrites are reproducible helpers in data-raw/helpers.R (canonicalize_silverchair_url, decode_scholar_redirect), so re-running data_processing.R keeps the links stable.