WIP: Results verification #86
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a script `csv_to_json` which is essentially a wrapper for `csvjson` found in the python3-csvkit package. This is the first step for JSON results in Zathras, but for now this is intended to convert the current CSV files to a results JSON (metadata is stripped) so we can easily verify those results. Co-developed-by: Cursor
This adds `verify_results` which uses the pydantic python module to verify if results match certain constraints, akin to JSON schemas. This script dynamically imports a passed script and a passed class then verifies if the JSON file/input matches. Co-developed-by: Cursor
This adds `--verify_skip` and `--json_skip` which skips the verification step and json conversion step respectively. This is placed inside of general_setup so the scripts can simply pass $to_json_flags and $to_verify_flags to avoid having to manage an ever-growing pile of flags in the future. Co-developed-by: Cursor
This adds `$to_script_dir` so it is easy for running scripts to fetch the directory that the script lives in. This will make verification scripts a lot easier to manage. Co-developed-by: Cursor
Previously the `verify_results` script could not handle absolute paths causing import errors and validation to fail. This changes the import logic so we can import a file from anywhere on a system and load the relevant class we need. Co-developed-by: Cursor
|
Bug: csvjson casts ints to floats. |
Previously we used csvkit to convert the csv files to JSON, but csvkit does not keep integer fields as integers. It will convert integers to floats, which will fail validation processes. The change to pandas will respect data types. Co-developed-by: Cursor
…ers into feat/result-verify Co-developed-by: Cursor
Change args.output to args.output_file, since I updated the arg at one point and forgot to do it everywhere else, d'oh Co-developed-by: Cursor
This lets pandas know that a line starting with a # is a comment and should be ignored. Co-developed-by: Cursor
This commit introduces the ability for verify_results and csv_to_json to install their needed dependencies if it is not available on the system. csv_to_json installs pandas, verify_results installs pydantic Co-developed-by: Cursor
This simplifies the default arguments so argparse takes care of opening the files with the needed modes, and when the default is stdout/stdin no other logic needs to be done to swap between file and console IO. Co-developed-by: Cursor
|
Converted to using pandas instead of csvkit |
dvalinrh
requested changes
Sep 2, 2025
| @@ -200,6 +202,14 @@ do | |||
| --usage) | |||
| gs_usage_info | |||
| ;; | |||
| --verify_skip) | |||
Change --schema_file type to a str, since we never directly read from the file (this is handled by importlib), and the necessary errors are already handled.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds 2 scripts
csv_to_jsonandverify_resultscsv_to_jsonconverts CSV files to JSON files using python's csvkit moduleverify_resultsimports a pydantic model dynamically and then verifies wheter a passed JSON file matches the schema provided.Before/After Comparison
Before
We were using regular expressions to try and validate results, which are error-prone, fragile, and annoying to maintain
After
we use pydantic models to easily say "this field is greater than 0, and is a floating point number"
Clerical Stuff
Issue TBD
Relates to JIRA: RPOPC-<TICKET_NUMBER>