Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify Logic of Data Loading and validation #8

Open
valentinedwv opened this issue Jul 7, 2022 · 3 comments
Open

Specify Logic of Data Loading and validation #8

valentinedwv opened this issue Jul 7, 2022 · 3 comments
Assignees

Comments

@valentinedwv
Copy link
Contributor

There is a validation notebook prototypes
There are site map, sparql, s3 and other counts shown

Put the existing and future logic into a document. This will be the basis for discussing the what parts of the validation are implemented, and how we test them.
It it's a living document (aka notebook), please add full descriptions of the steps, then implement.
Aka test description first, code second.

@MBcode
Copy link
Contributor

MBcode commented Jul 25, 2022

Here is a markdown version with more diagrams as well
I will try to sync it up w/the other issue's/doc-outlines as well

@MBcode
Copy link
Contributor

MBcode commented Sep 12, 2022

This turned into testing.md that started with utils validation section, that was split out to spot_ crawl_dropoff which can also have csv_dropoff 10

@MBcode
Copy link
Contributor

MBcode commented Sep 12, 2022

validation can be iterated on for awhile, incl the description, which will be refined
So we might want to break this up in stages, so we can end the initial one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants