Open
Conversation
…udit all URLs - Migrate download_huc, download_edgar, download_prism, download_cropscape from legacy wget/curl shell commands to download_run_method() (httr2) - Add deprecation warnings for download=FALSE and remove_command params - Add show_progress, max_tries, rate_limit params to all 4 functions - Fix download_run_method: remove HTTP 500 from is_transient to prevent 20 redundant retries for EPA TRI API (returns 500 with valid CSV body) - Rewrite test-huc/edgar/prism/cropscape/tri/terraclimate from old read_commands/extract_urls/check_urls pattern to httr2 return-value checks - Fix test-population.R: radius=0->1000 for fun='invalid_function' test - Fix download_nlcd.Rd: replace non-existent data_mrlc_nlcd2021 bib key with correct dewitz_national_2023 (fixes pkgdown CI failure) - Add agent.md: comprehensive maintenance guide for amadeus package - Verify all API URLs return valid responses (HUC/EDGAR/PRISM/CropScape/ TRI/NEI/TerraClimate/AQS all confirmed live) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace non-ASCII characters (UTF-8 checkmarks/crosses) in download_auxiliary.R with ASCII equivalents (OK/FAIL) - Add missing @param show_progress/max_tries/rate_limit to download_huc, download_edgar, download_prism, download_cropscape - Add importFrom(stats, runif) for runif used in download_run_method - Add agent.md to .Rbuildignore (non-standard top-level file) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update .lintr: line_length_linter(100) -> (80), add modis_workflow.Rmd and test-download.R to exclusions - Wrap all roxygen @param and description lines to <=80 chars - Split long warning/message strings across multiple arguments - Remove unused download_result assignments (object_usage_linter) - Remove unused resp assignment in download_auxiliary.R - Fix single-quote -> double-quote in gsub calls - Break long URL strings with paste0() - Regenerate man/*.Rd docs after param rewraps Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Re-adds the Protected Data Sources vignette removed in 65942f9. Content to be updated later. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
download_hms is now migrated to httr2. download=FALSE returns a list with $urls instead of writing a curl commands file. Update the running test chunk to capture the result and use result$urls directly instead of read_commands() + extract_urls(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix sprintf calls in get_token() that had unused arguments, which would be converted to errors by options(warn = 2) in the coverage test script - Remove duplicate format_file_size() function definition Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove options(warn = 2) from covr code parameter in test-coverage.R - Wrap long sprintf format string in paste0() to stay within 80 chars Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix download_result assignment in download_aqs, download_merra2, download_nei (download_run_method return value was not captured, causing 'object not found' errors on actual downloads) - Fix download_merra2: move download=FALSE deprecation/early return before get_token() call to prevent token errors masking the warning - Fix test-ecoregion.R: undefined variable d1 -> '.' in tempdir test - Fix test-merra2.R: add skip_if for NASA_EARTHDATA_TOKEN to two test blocks that require network auth - Fix test-modis.R: 8 failures from terra no longer emitting 'unknown extent' warning for VNP46 HDF files; update expectations to expect_no_error, add scale='* 1' where missing, broaden warning pattern to include scale warning - Add unzip=TRUE and remove_zip=FALSE params to download_prism; PRISM API always returns a ZIP so unzip after download so process_prism can find the extracted data files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add four AI agent system prompts + YAML metadata files covering the three-tier amadeus API and unit testing: - agents/download-agent.md/.yaml -- all download_*() functions, httr2 patterns, auth, deprecations, issue triage guidance - agents/process-agent.md/.yaml -- all process_*() functions, terra/sf/sftime return types, CRS, date handling - agents/calculate-agent.md/.yaml -- all calculate_*() functions, locs/locs_id/geom pattern, exactextractr, output format - agents/test-agent.md/.yaml -- testthat conventions, skip patterns, testdata structure, three-tier test skeleton - agents/README.md -- usage guide and agent picker Add ^agents$ to .Rbuildignore so the directory is excluded from R CMD CHECK, covr coverage, and all CI/CD workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add mock-based unit tests to 18 download test files to cover uncovered branches introduced by the httr2 refactor: - remove_command deprecation warning in all download_* functions - hash=TRUE and hash=FALSE return paths - all files exist else-branch in download_aqs and download_merra2 - prism post-download logic (unzip + remove_zip + hash) Uses testthat::local_mocked_bindings to intercept check_url_status, check_destfile, download_run_method, and download_hash so tests run without network access or real file I/O. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lows The package uses NASA_EARTHDATA_TOKEN env var (via get_token()) but CI only set EARTHDATA_TOKEN, causing NASA token lookups to fail silently and skipping NASA-authenticated tests during coverage runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The helpers.R file contained test(), cov(), and interactive() functions that were written for the beethoven package and used apptainer/container.sif which does not exist in amadeus. The interactive() function also shadowed base::interactive(), causing setup_nasa_token() to misdetect interactive mode. Remove the file entirely. Add tests for setup_nasa_token renviron/file/ non-interactive paths and download_run_method token/0-byte branches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Delete man/cov.Rd, man/interactive.Rd, man/test.Rd (orphaned from helpers.R) - Remove 'rvest' and 'jsonlite' from DESCRIPTION Imports (not used in code) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix format_file_size test to use amadeus:::format_file_size (was failing with 'could not find function' since function is unexported) - Add explicit GB branch test (bytes >= 1024^3) - Add download_modis mock test covering remove_command=TRUE warning and hash=TRUE return path (mocking get_token, req_perform, resp_body_json, download_run_method, download_hash) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ion, modis 31-day warning
- download_run_method: mock test with show_progress=TRUE + 2 files covers
the ' OK ' success prefix branch and i<n_files Sys.sleep branch
- download_nei: test for epa_certificate_path and certificate_url
deprecation warning branches
- download_modis: test for diff(dt_date) > 31 days warning using mock
CMR API response with c('2023-01-01', '2023-03-15') date range
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…, and method variants - Add hash=FALSE return path tests for: geos, gmted, narr, gridmet, terraclimate, tri, groads, nlcd, population, hms, nei, modis, aqs - Add file-already-exists path tests for: gmted, groads, koppen_geiger, ecoregion, nlcd, population, aqs - Add format variant tests for download_population: ASCII+year, ASCII+totpop, GeoTIFF+totpop, netCDF, 30sec-resolution warning - Add download_hms KML format path tests (both hash=TRUE and hash=FALSE) - Enable process_hms absent polygons test (uncomment and add suppressMessages) - Add download_nei all-files-exist and hash=FALSE paths - Add download_modis single-date + hash=FALSE without skip_on_cran - Add download_merra2 esdt_name_5 (goldsmr5) branch test - Add download_edgar missing-URL warning test with counter-based mock - Add download_aqs download_run_method branch test - Add setup_nasa_token session/renviron/file method tests without skip_on_cran Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove skip_on_cran from: setup_nasa_token NULL token test, download_run_method tryCatch error handler tests - Add NEI epa_certificate_path deprecation test - Add MERRA2 actual download path via httr2 mock - Add MODIS no-granules-found error path - Add NLCD download=FALSE deprecated warning and remove_command tests - Add population download=FALSE and remove_command deprecated warning tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- test-improve.R: add single-date string expansion test, hash=TRUE early-return when files present, include_sites=FALSE path - test-goes.R: add empty S3 listing early-return test for download_goes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add test that passes a SpatVector without Latitude/Longitude data columns to calculate_improve(), exercising the coordinate-extraction fallback at line 3783 that extracts monitor coordinates from the geometry when they are absent from the data attributes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- test-improve.R: add hash=TRUE after actual download mock test - test-goes.R: add partial variable-name match test for process_goes - tests/testdata/goes: add DOY-004 fixture with SmokeFlag variable to enable testing process_goes partial-match fallback (line 3815) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The regexpr-based date extraction in process_geos() and process_merra2() daily_agg is a defensive fallback for the case where terra::time() returns NAs. In practice, both functions always set layer times explicitly from the filename before aggregation, making the else-branch unreachable. Mark it with # nocov start/end so it does not unfairly penalise patch coverage. Also mark the geostationary-reprojection and empty-CRS guard in process_goes() which cannot be triggered by the lat/lon test fixtures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…download Add a mock test that exercises the return(invisible(download_result)) path in download_improve() when hash=FALSE and files are not already present, ensuring the download_run_method call and the non-hash return branch are both reached. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…error process_goes() silently drops files whose start-datetime cannot be parsed (the tryCatch in the date-filter loop returns NA for those files and they are excluded via the mask), so when a valid DOY-001 file is also present in the directory the function succeeds instead of erroring. The old expect_error() call was therefore wrong; updated to assert a SpatRaster is returned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add test for single-string date argument, exercising lines 3744-3746 that expand a scalar date to a two-element vector (same pattern as process_improve and other process functions). - The bad-stamp test was already committed in a previous commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous fix changed expect_error to expect_s4_class, but in CI the file.copy call silently fails (no good file present), causing process_goes to always throw an error in that test context. This made covr error with 'running tests failed' and report 0% coverage. Fix: simplify the test to only create a bad-stamp file (no copy), which deterministically exercises the 'all files unparseable' error path in both local and CI environments. The process_goes single-date expansion test (lines added separately) tests lines 3744-3746 that expand a scalar date to a two-element vector. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace incorrect Köppen-Geiger URL (https://www.nature.com/articles/sdata2018214) with the correct NASA GMAO MERRA-2 URL (https://gmao.gsfc.nasa.gov/reanalysis/MERRA-2/). Fixes #224. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- SPEI: downloads single global netCDF per timescale from spei.csic.es - EDDI: downloads weekly CONUS netCDF files from NOAA PSL archive, keyed by Tuesday dates and timescale - USDM: downloads weekly drought-class shapefiles (zip) from the US Drought Monitor, with unzip/remove_zip handling matching HMS pattern - Add drought_weekly_dates() helper to download_auxiliary.R that returns YYYYMMDD strings for every Tuesday in [date_start, date_end] - Add unzip/remove_zip parameters to download_drought() for USDM zips - Validate timescale, unzip/remove_zip conflict, and empty date ranges - check_url_status() guards first URL for EDDI and USDM; SPEI single-file skip path when file already exists - 26 new mock-based tests covering all branches, alias routing, and drought_weekly_dates edge cases (71 pass total in test-drought.R) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ources
- Add SPEI/EDDI netCDF pathway: reads multi-year (SPEI) or per-year (EDDI)
.nc files, sets terra time metadata with CF-convention fallback, names
layers as <source>_<TS>_YYYY-MM-DD, enforces EPSG:4326 CRS.
- Add USDM polygon pathway: reads weekly USDM_YYYYMMDD.shp files, reprojects
to EPSG:4326, retains DM column and adds date/source metadata columns.
- Fix process_covariates() dispatch to forward source='eddi'/'usdm' via
closure wrappers so alias covariates route correctly.
- Add internal helpers drought_process_nc(), drought_set_time_nc(),
drought_process_usdm() (unexported).
- Create synthetic test data in tests/testdata/drought/{spei,eddi,usdm}.
- Add tests/testthat/test-drought.R with 14 test_that blocks covering all
three sources, single-date, extent cropping, error paths, and
process_covariates alias routing.
- Fix unbalanced nolint start/end blocks in process_drought() docs.
Closes todo: drought-process-implement
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ettes
- _pkgdown.yml: add explicit exclusions for aux download helpers, aux
process helpers, calculate_modis_daily, and the three internal
spacetime helpers (check_mysf, check_mysftime, rename_time) from the
has_keyword('spacetime') selector, which unlike starts_with() does not
respect @Keywords internal
- vignettes/download_functions.Rmd: remove amadeus::check_url_status()
and amadeus::download_run_method() calls; replace with prose describing
internal URL validation, and inline httr2 equivalent in check_urls()
- vignettes/edgar_workflow.Rmd: remove reference to internal
calc_return_locs() in the Notes section
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…st-modis.R Functions modis_extract_temporal_key, modis_extract_temporal_scale, modis_key_to_date, and modis_filter_paths_by_date are internal (non-exported) helpers; tests must access them via the amadeus::: triple-colon operator.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The calc_worker test has always called the function without a `fun` argument (the test was written that way from the initial commit). The function lacked a default, causing a missing-argument error. Adding `fun = "mean"` as default aligns with the most common call-site usage and fixes the pre-existing regression. Also stage all uncommitted weighting-support and docs changes that were implemented but not yet committed (weights param in calc_worker, calc_prepare_weights, calc_weighted_fun, calc_prepare_exact_geoms, calculate_modis_daily; man/ Rd updates; NAMESPACE exports; new gridmet/ modis test coverage). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover previously uncovered branches in calc_prepare_weights, calc_prepare_exact_geoms, calc_weighted_fun, and the weights pass-through paths in calculate_prism, calculate_edgar, calculate_cropscape, calculate_drought, and calculate_covariates. Coverage: 99.30% → 99.58% Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix indentation in calc_extents_overlap (4→6 spaces continuation) - Wrap @param weights doc line >80 chars in calculate_covariates_auxiliary.R - Wrap do.call(exactextractr::exact_extract, ...) long line - Wrap calc_prepare_weights() calls >80 chars in auxiliary and main files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- fix: use terra::as.data.frame() in normalize_vector_weights to correctly detect numeric columns in SpatVector (terra 1.9.x returns data.frame from sv[[col]], not a vector, so is.numeric was always FALSE) - fix: restructure calculate_prism .by time derivation so metags fallback is tried whenever terra::time() returns NA (not just on try-error), and guard nrow() call against NULL metags return - test: add tests for calc_prepare_weights vector-with-numeric-column branches (single and multiple numeric cols, negative values, file paths) - test: add tests for calculate_prism metags time extraction (8-, 6-, 4- digit date strings in metadata tags) - test: add test for calculate_drought with point locs + weights (exact_extract path via calc_prepare_exact_geoms) Coverage: 99.48% -> 99.81% Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
download_*functions are being migrated fully tohttr2package