Release 0.2.0: ploswater dataset and washdev update to mid-2026

.Rbuildignore

@@ -9,4 +9,4 @@
 ^docs$
 ^pkgdown$
 ^\.github$
-^inst
+^python-scripts-lesson\.md$

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: washopenresearch
 Title: Dataset about open research data information in Water, Sanitation, and Hygiene
-Version: 0.1.0
+Version: 0.2.0
 Authors@R: c(
     person("Mian", "Zhong", , "mzhong@ethz.ch", role = c("aut", "cre"),
            comment = c(ORCID = "0009-0009-4546-7214")),
@@ -9,15 +9,15 @@ Authors@R: c(
     person("Lars", "Schöbitz", , "lschoebitz@ethz.ch", role = "aut",
            comment = c(ORCID = "0000-0003-2196-5015"))
   )
-Description: The goal of washopenresearch is to provide an overview of open research data related to Water Sanitation and Hygiene (WASH). The package provides access to two datasets `washdev` and `uncnewsletter`. Each dataset collects information on scientific articles about (1) article metadata (e.g. title, first author, correspondence author), (2) supplementary material information, (3) data availability statement, and (4) semantic information (e.g. keywords).
+Description: The goal of washopenresearch is to provide an overview of open research data related to Water Sanitation and Hygiene (WASH). The package provides access to three datasets `washdev`, `uncnewsletter`, and `ploswater`. Each dataset collects information on scientific articles about (1) article metadata (e.g. title, first author, correspondence author), (2) supplementary material information, (3) data availability statement, and (4) semantic information (e.g. keywords).
 License: CC BY 4.0
 Encoding: UTF-8
 Roxygen: list(markdown = TRUE)
 Depends: 
     R (>= 3.5)
 LazyData: true
 Config/Needs/website: rmarkdown
-Date: 2026-07-07
+Date: 2026-07-08
 URL: https://github.com/openwashdata/washopenresearch
 BugReports: https://github.com/openwashdata/washopenresearch/issues
 Config/roxygen2/version: 8.0.0

NEWS.md

@@ -1,3 +1,25 @@
+# washopenresearch 0.2.0
+
+## New features
+
+- New dataset `ploswater` with all 436 articles of the journal PLOS Water from its first volume (2022) to July 2026, collected through the public PLOS API (#14). Beyond the shared schema it records `das_repo_url` (links and dataset DOIs in the data availability statement), `das_repo_name` (the recognized repository behind them), `article_type`, and `publication_date` (#15). Data availability statements are mandatory at PLOS: all 333 research articles carry one, and 177 articles link out to a data location.
+- `washdev` now covers volume 1 (2011) through volume 16 issue 6 (June 2026) with 1173 observations, up from 932 (#12). The 241 new rows cover volumes 14 to 16.
+- `washdev` gains a `doi` column, filled for the newly scraped articles; earlier rows will be backfilled via Crossref (#20).
+- Data acquisition is now fully R. The washdev scraper was ported from Python/Selenium to `data-raw/washdev_scraping.R` using chromote and rvest, with incremental updates (#11). The Python tooling in `inst/python/`, including a 17 MB chromedriver binary, was removed (#17).
+
+## Bug fixes
+
+- The manual supplement-type corrections for `uncnewsletter` were indexed against `washdev` paperids and landed on the wrong rows, and `correspondence_author_affiliation_country` was never cleaned because the cleaned values were written into `first_author_affiliation_country`, overwriting it. Both are fixed and `uncnewsletter` was regenerated; country values changed on 56 rows and supplement types on 16 rows (#13).
+- Two author names in `washdev` (for example "Inês Freire Machete") carried Mac Roman bytes that made the xlsx export fail; the raw data is repaired at read time (#13).
+- `.Rbuildignore` excluded the whole `inst/` directory from the built package, so `citation("washopenresearch")` and the `inst/extdata` files were missing from installed packages. The rule is now scoped correctly (#17).
+
+## Minor improvements
+
+- The data dictionary and roxygen documentation use the actual variable names `first_author_affiliation_country` and `correspondence_author_affiliation_country` for `washdev` (previously documented as `*_affiliation_region`) and document the `url_source` and `doi` variables. The `uncnewsletter` dictionary entry for `issue_url` is corrected from "Volume number of the journal" (integer) to the newsletter issue URL (character).
+- `uncnewsletter` is documented as a frozen source: the newsletter ceased publication in May 2024 (#17).
+- For newly scraped `washdev` rows, mixed supplementary file types are recorded as " & "-joined lists (one type per file) instead of the literal "misc", which previously required manual repair.
+- Data values that no cleaning rule could resolve are written to review files under `data-raw/` (`*-das-review.csv`, `*-country-review.csv`) instead of being fixed by hand, so every correction stays in reproducible R code (#12, #15).
+
 # washopenresearch 0.1.0
 
 ## Breaking changes

R/ploswater.R

@@ -0,0 +1,47 @@
+#' Dataset about data availability in PLOS Water
+#'
+#' Article metadata for all articles published in PLOS Water since its
+#' first volume (2022), collected through the public PLOS search API and
+#' each article's JATS XML. Data availability statements are mandatory at
+#' PLOS, so `has_das` is TRUE for research articles throughout; the
+#' interesting variation lies in `das_type`, `das_repo_url`, and
+#' `das_repo_name`. All article types are included; use `article_type`
+#' to restrict to research articles.
+#'
+#' @format ## `ploswater`
+#'
+#' \describe{
+#'   \item{paperid}{DOI of the paper, identical to the doi variable}
+#'   \item{volume}{Volume number of the journal; volume 1 is 2022}
+#'   \item{issue}{Issue number of the journal}
+#'   \item{paper_url}{Official website url of the paper}
+#'   \item{journal}{Full name of the journal}
+#'   \item{title}{Title of the paper}
+#'   \item{published_year}{Year of publication}
+#'   \item{is_supp}{Whether the paper has supplementary materials}
+#'   \item{num_supp}{Number of supplementary material files}
+#'   \item{supp_file_type}{File types of the supplementary materials, separated by "; " when there are multiple}
+#'   \item{supp_url}{Website urls of the supplementary materials, separated by "; " when there are multiple. These links are stable download endpoints, they do not expire.}
+#'   \item{num_authors}{Number of the authors}
+#'   \item{first_author_name}{Name of the first author}
+#'   \item{first_author_affiliation}{Academic affiliation of the first author}
+#'   \item{first_author_affiliation_country}{Country of the first author parsed from first_author_affiliation, encoded with United Nations names}
+#'   \item{first_author_email}{Email of the first author. Only available when the first author is also the correspondence author, because PLOS publishes only the correspondence email.}
+#'   \item{first_author_orcid}{ORCID of the first author}
+#'   \item{correspondence_author_name}{Name of the correspondence author}
+#'   \item{correspondence_author_affiliation}{Academic affiliation of the correspondence author}
+#'   \item{correspondence_author_affiliation_country}{Country of the correspondence author parsed from correspondence_author_affiliation, encoded with United Nations names}
+#'   \item{correspondence_author_email}{Email of the correspondence author}
+#'   \item{correspondence_author_orcid}{ORCID of the correspondence author}
+#'   \item{has_das}{Whether the paper has a data availability statement}
+#'   \item{das}{Original data availability statement of the paper. NA if it does not have a data availability statement.}
+#'   \item{das_type}{Type of the data availability statement including available in online repository(data is shared in a public online repository) in paper(data in full paper scope like supplementary material or appendix or main content) on request(data available on request to the authors) not shareable(data is not shareable) no data generated(the study produced no datasets). NA if it does not have a data availability statement or no classification rule matched.}
+#'   \item{das_repo_url}{Website urls and dataset DOIs mentioned in the data availability statement, separated by "; " when there are multiple}
+#'   \item{das_repo_name}{Recognized data repositories behind das_repo_url (e.g. zenodo, dryad, figshare, osf, github, dataverse), separated by "; " when there are multiple}
+#'   \item{keywords}{Subject terms of the paper from the PLOS search API, separated by "; ". PLOS Water articles carry no author keywords in their XML.}
+#'   \item{url_source}{Publisher website of the paper}
+#'   \item{doi}{DOI of the paper}
+#'   \item{article_type}{Article type, e.g. "Research Article", "Opinion", "Review"}
+#'   \item{publication_date}{Date of publication (ISO 8601)}
+#' }
+"ploswater"

R/washdev.R

@@ -17,18 +17,20 @@
 #'   \item{num_authors}{Number of the authors}
 #'   \item{first_author_name}{Name of the first author}
 #'   \item{first_author_affiliation}{Academic affiliation of the first author}
-#'   \item{first_author_affiliation_region}{Country or region of the first author parsed from first_author_affiliation variable}
+#'   \item{first_author_affiliation_country}{Country of the first author parsed from first_author_affiliation, encoded with United Nations names}
 #'   \item{first_author_email}{Email of the first author}
 #'   \item{first_author_orcid}{ORCID of the first author}
 #'   \item{correspondence_author_name}{Name of the correspondence author}
 #'   \item{correspondence_author_affiliation}{Academic affiliation of the correspondence author}
-#'   \item{correspondence_author_affiliation_region}{Country or region of the correspondence author parsed from correspondence_author_affiliation variable}
+#'   \item{correspondence_author_affiliation_country}{Country of the correspondence author parsed from correspondence_author_affiliation, encoded with United Nations names}
 #'   \item{correspondence_author_email}{Email of the correspondence author}
 #'   \item{correspondence_author_orcid}{ORCID of the correspondence author}
 #'   \item{has_das}{Whether the paper has a data availability statement}
 #'   \item{das}{Original data availability statement of the paper.  NA if it does not have a data availability statement.}
 #'   \item{das_type}{Type of the data availability statement including in paper(data in full paper scope like supplementary material or appendix or main content) on request(data available on request to the authors) available in online repository(data is shared in a public online repository) not shareable(data is not shareable). NA if it does not have a data availability statement.}
 #'   \item{das_repo_url}{Website urls of the data if the relevant data of the paper is shared on a public repository, separated by "; " when there are multiple}
 #'   \item{keywords}{Keywords of the paper, separated by "; "}
+#'   \item{url_source}{Publisher website of the paper}
+#'   \item{doi}{DOI of the paper. Collected since the R port of the scraper; NA for articles scraped earlier, to be backfilled via Crossref.}
 #' }
 "washdev"

README.Rmd

@@ -38,13 +38,15 @@ library(wordcloud2)
 
 The goal of washopenresearch is to provide an overview of open research
 data related to Water Sanitation and Hygiene (WASH). The current version
-contains two datasets from the following sources:
+contains three datasets from the following sources:
 
 -   `washdev`: Open access journal [*Journal of Water, Sanitation and
     Hygiene for Development*](https://iwaponline.com/washdev)
 -   `uncnewsletter`: Research section of the newsletter [North Carolina
     Water
     News](https://waterinstitute.unc.edu/our-work/nc-water-news-newsletter)
+-   `ploswater`: Open access journal [*PLOS
+    Water*](https://journals.plos.org/water/)
 
 ![Word cloud of the most frequent keywords in articles of the Journal of Water, Sanitation and Hygiene for Development, with water, sanitation, and hygiene appearing largest](man/figures/washdev_wordcloud.png){width="515"}
 
@@ -79,11 +81,12 @@ read_csv("data-raw/dictionary.csv") |>
 
 ## Data
 
-The package provides access to two datasets `washdev` and
-`uncnewsletter`. Each dataset collects information on scientific
-articles about (1) article metadata (e.g. title, first author,
-correspondence author), (2) supplementary material information, (3) data
-availability statement, and (4) semantic information (e.g. keywords).
+The package provides access to three datasets `washdev`,
+`uncnewsletter`, and `ploswater`. Each dataset collects information on
+scientific articles about (1) article metadata (e.g. title, first
+author, correspondence author), (2) supplementary material information,
+(3) data availability statement, and (4) semantic information (e.g.
+keywords).
 
 ```{r}
 library(washopenresearch)
@@ -92,9 +95,11 @@ library(washopenresearch)
 ### washdev
 
 The dataset `washdev` contains data on open access articles of the
-*Journal of Water, Sanitation & Hygiene for Development* (Vol.1 Issue
-1 - Vol.13 Issue 11). It has `r nrow(washdev)` observations from March
-2011 to November 2023.
+*Journal of Water, Sanitation & Hygiene for Development* (Vol. 1
+Issue 1 to Vol. `r max(washdev$volume)` Issue
+`r max(washdev$issue[washdev$volume == max(washdev$volume)])`). It has
+`r nrow(washdev)` observations from March 2011 to
+`r max(washdev$published_year)`.
 
 ```{r}
 washdev |> 
@@ -119,6 +124,8 @@ readr::read_csv("data-raw/dictionary.csv") |>
 The dataset `uncnewsletter` contains data on a curated list of articles
 published at the Research section of the newsletter North Carolina Water
 News. It has `r nrow(uncnewsletter)` observations from 2020 to 2023.
+The newsletter ceased publication in May 2024, so this dataset is a
+frozen source.
 
 ```{r}
 uncnewsletter |> 
@@ -132,12 +139,40 @@ For an overview of the variable descriptions, see the following table.
 ```{r echo=FALSE, message=FALSE, warning=FALSE}
 readr::read_csv("data-raw/dictionary.csv") |>
   dplyr::filter(file_name == "uncnewsletter.rda") |>
-  dplyr::select(variable_name:description) |> 
-  knitr::kable() |> 
-  kableExtra::kable_styling("striped") |> 
+  dplyr::select(variable_name:description) |>
+  knitr::kable() |>
+  kableExtra::kable_styling("striped") |>
   kableExtra::scroll_box(height = "200px")
 ```
 
+### ploswater
+
+The dataset `ploswater` contains data on all articles of the journal
+[*PLOS Water*](https://journals.plos.org/water/) from its first volume
+(2022) onward, collected through the public PLOS API rather than web
+scraping. It has `r nrow(ploswater)` observations. Data availability
+statements are mandatory at PLOS, so the interesting variation lies in
+`das_type`, `das_repo_url`, and `das_repo_name`, which describe how and
+where the data behind each article is stored. All article types are
+included; use `article_type` to restrict to research articles.
+
+```{r}
+ploswater |>
+  head(3) |>
+  gt::gt() |>
+  gt::as_raw_html()
+```
+
+For an overview of the variable descriptions, see the following table.
+
+```{r echo=FALSE, message=FALSE, warning=FALSE}
+readr::read_csv("data-raw/dictionary.csv") |>
+  dplyr::filter(file_name == "ploswater.rda") |>
+  dplyr::select(variable_name:description) |>
+  knitr::kable() |>
+  kableExtra::kable_styling("striped") |>
+  kableExtra::scroll_box(height = "200px")
+```
 
 ## Example