Summary
update_access() (R/update_access.R, currently on the 1.0.2.9000 dev source) produces wrong rows in data/metadata/access.csv for any package with more than one dataset, and for datasets whose export is not a csv+xlsx pair. It only works by accident for single-dataset packages whose dataset name equals the package name.
Found while converting jmpdata (two datasets: jmpraw, jmpindicators; the indicators export is jmpindicators.csv.gz only).
Observed
With a dictionary listing jmpraw.rda and jmpindicators.rda, update_access() wrote:
fileName,name,contentUrl,encodingFormat
jmpindicators.csv,jmpindicators,https://raw.githubusercontent.com/openwashdata/jmpindicators/main/inst/extdata/jmpindicators.csv,csv
jmpraw.csv,jmpraw,https://raw.githubusercontent.com/openwashdata/jmpraw/main/inst/extdata/jmpraw.csv,ooxml
jmpindicators.xlsx,jmpindicators,https://raw.githubusercontent.com/openwashdata/jmpindicators/main/inst/extdata/jmpindicators.xlsx,csv
jmpraw.xlsx,jmpraw,https://raw.githubusercontent.com/openwashdata/jmpraw/main/inst/extdata/jmpraw.xlsx,ooxml
Three problems:
- The repo segment of every URL is the dataset name (
openwashdata/jmpraw, openwashdata/jmpindicators), not the package repo (openwashdata/jmpdata). All four URLs 404.
encodingFormat is misaligned: jmpraw.csv is labelled ooxml and jmpindicators.xlsx is labelled csv.
jmpindicators.xlsx does not exist; the dataset ships as jmpindicators.csv.gz (about 38 MB uncompressed, so no xlsx). The function invents rows instead of reading inst/extdata/.
Root cause (R/update_access.R)
- Lines 35-36 build
contentUrl with paste0(".../openwashdata/", file_name, "/main/..."), so the dataset name stands in for the repo name. Single-dataset packages named after their dataset mask this.
- Line 32 builds
fileName as c(paste0(file_name, ".csv"), paste0(file_name, ".xlsx")) (length 2n), while line 38 sets encodingFormat = c("csv", "ooxml") (length 2). R recycles the short vector, which aligns correctly only for n = 1. For n = 2 the formats interleave wrongly, exactly as observed.
- The csv/xlsx pair is assumed rather than derived from the files actually present in
inst/extdata/.
- Line 18 defines an unused
extdata_path hardcoding the worldhdi repo; dead code from the same single-package assumption.
Suggested fix
- Derive the repo name from the package, for example
desc::desc_get("Package") or the DESCRIPTION URL field, instead of file_name.
- Enumerate
list.files("inst/extdata"), match files to datasets by name prefix, and map extension to encodingFormat per file (csv/csv.gz to csv, xlsx to ooxml). That fixes the recycling misalignment and the invented rows in one move, and picks up compressed exports.
- Drop the unused
extdata_path.
For jmpdata, access.csv was corrected by hand in openwashdata/jmpdata@770210b, which shows the expected output.
Possibly fits the v1.1.0 milestone (consolidate the FAIR layer).
Summary
update_access()(R/update_access.R, currently on the 1.0.2.9000 dev source) produces wrong rows indata/metadata/access.csvfor any package with more than one dataset, and for datasets whose export is not a csv+xlsx pair. It only works by accident for single-dataset packages whose dataset name equals the package name.Found while converting jmpdata (two datasets:
jmpraw,jmpindicators; the indicators export isjmpindicators.csv.gzonly).Observed
With a dictionary listing
jmpraw.rdaandjmpindicators.rda,update_access()wrote:Three problems:
openwashdata/jmpraw,openwashdata/jmpindicators), not the package repo (openwashdata/jmpdata). All four URLs 404.encodingFormatis misaligned:jmpraw.csvis labelledooxmlandjmpindicators.xlsxis labelledcsv.jmpindicators.xlsxdoes not exist; the dataset ships asjmpindicators.csv.gz(about 38 MB uncompressed, so no xlsx). The function invents rows instead of readinginst/extdata/.Root cause (R/update_access.R)
contentUrlwithpaste0(".../openwashdata/", file_name, "/main/..."), so the dataset name stands in for the repo name. Single-dataset packages named after their dataset mask this.fileNameasc(paste0(file_name, ".csv"), paste0(file_name, ".xlsx"))(length 2n), while line 38 setsencodingFormat = c("csv", "ooxml")(length 2). R recycles the short vector, which aligns correctly only for n = 1. For n = 2 the formats interleave wrongly, exactly as observed.inst/extdata/.extdata_pathhardcoding theworldhdirepo; dead code from the same single-package assumption.Suggested fix
desc::desc_get("Package")or the DESCRIPTION URL field, instead offile_name.list.files("inst/extdata"), match files to datasets by name prefix, and map extension toencodingFormatper file (csv/csv.gz tocsv, xlsx toooxml). That fixes the recycling misalignment and the invented rows in one move, and picks up compressed exports.extdata_path.For jmpdata,
access.csvwas corrected by hand in openwashdata/jmpdata@770210b, which shows the expected output.Possibly fits the v1.1.0 milestone (consolidate the FAIR layer).