Update washdev with volumes 14-16 (2024 to mid-2026)

[larnsce]

Background

data-raw/washdev.csv currently covers volumes 1-13 (2011-2023), 931 articles, with volume 13 complete through issue 12. The journal publishes monthly, so as of July 2026 the dataset is missing volume 14 (2024, 12 issues), volume 15 (2025, 12 issues), and volume 16 (2026, issues published to date).

Tasks

• Run the scraper (see Port washdev scraper from Python/Selenium to R with incremental updates #11) for volumes 14, 15, and 16 to date, and append the results to data-raw/washdev.csv.
• Re-run data-raw/data_processing.R end to end.
• Triage the new rows that need manual fixes, the same way earlier volumes did:
  • affiliation strings whose country could not be parsed (the manual paperid fix-up step),
  • new das_type phrasings that the regex rules do not yet map to the factor levels,
  • supplementary file types that come out as "misc" and need manual typing.
• Regenerate data/washdev.rda and the exports in inst/extdata/.

Notes

• Depends on Port washdev scraper from Python/Selenium to R with incremental updates #11 (or on patching the existing Selenium script).
• Related: supp_url and paper_url contain expired pre-signed CDN links and Scholar redirect URLs #10 reports that supp_url contains expired pre-signed CDN links. A re-scrape refreshes those links but does not fix the underlying problem; keep the two issues separate.

[larnsce]

Triage the new rows that need manual fixes

Do not do any manual fixes without prior approval. Every fix should be covered in R code for full reproducibility.

[larnsce]

Volumes 14-16 are scraped and committed on dev: 241 new articles (vol 14: 113, vol 15: 86, vol 16 through issue 6: 42), all with DOIs, zero scrape failures. washdev is now 1173 observations, 2011 to June 2026. Per the reproducibility rule, nothing was fixed by hand; two review files and three proposals instead:

1. das_type leftovers (data-raw/washdev-das-review.csv, 14 rows)

• 7 rows from volume 13 issue 12 (the last Python-scraped batch) have has_das = TRUE but no DAS text; the old scraper found the heading but failed to extract the body. Proposal: re-scrape these 7 articles with the R scraper (data collection, not a hand edit) and patch the raw rows by paperid in code.
• 5 of the 7 new-row leftovers would be resolved by applying the same broadened rule set I wrote for ploswater (Harmonize ploswater into the shared schema, including das_type classification #15) to washdev: "upon reasonable request" phrasings (3 rows), "provided as supplementary material" (1), NCBI SRA BioProject (1). Proposal: port those rules into the washdev das_type block.
• 1 case needs a human call: paperid 107295 pasted MDPI's data-availability template text as its DAS, verbatim including "please provide details". Proposal: leave das_type NA and enjoy the irony.
• 1 case (99802) lists five government/data-program repository links: available in online repository.

2. Country leftovers (data-raw/washdev-country-review.csv, 33 rows: 21 legacy rows the original manual pass never covered, 12 new)

Proposal: extend the parsing rules (strip trailing commas and postcodes before taking the last comma chunk, small alias map for e.g. "UK"), then bring whatever remains back here as a per-paperid table for approval.

Awaiting your go-ahead on the three proposals; then I regenerate and this closes.

[larnsce]

1 case needs a human call: paperid 107295 pasted MDPI's data-availability template text as its DAS, verbatim including "please provide details". Proposal: leave das_type NA and enjoy the irony.

take a #permanent note of this, highlight it in script, so that I can resurface in discussion for the manuscript.

[larnsce]

Shipped in 0.2.0 via PR #21 (commit 3d80f95).