Skip to content

define csv assets as package data #13

@alasdairwilson

Description

@alasdairwilson

We should define the csv assets here as package data like this:

[tool.setuptools.package-data]
isaricanalytics = ["assets/*.csv"]

then we can load them with importlib rather than brittle .parent.parent stuff in load_units_conversion_table and load_countries_table to be, you can keep the api fallback but this way the local version actually has a chance to work

def load_units_conversion_table() -> pd.DataFrame:
    try:
        asset = files("isaricanalytics.assets").joinpath("conversion_table.csv")
        return pd.read_csv(asset)
    except Exception:
        return pd.read_csv(
            "https://raw.githubusercontent.com/ISARICResearch/VERTEX/refs/heads/main/assets/conversion_table.csv"
        )

def load_countries_table(encoding: str = "latin-1") -> pd.DataFrame:
    try:
        asset = files("isaricanalytics.assets").joinpath("countries.csv")
        return pd.read_csv(asset, encoding=encoding)
    except Exception:
        return pd.read_csv(
            "https://raw.githubusercontent.com/ISARICResearch/VERTEX/refs/heads/main/assets/countries.csv",
            encoding=encoding,
        )

then update the tests

Test strategy should change too:

  • Tests should assert that the package loader reads packaged r resources.
  • Tests should stop pretending test fixtures are the production asset location.
  • mock the local file resolution to error and we can to test the remote fallback separately

Originally posted by @alasdairwilson in #8 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions