Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: UT #100

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

WIP: UT #100

wants to merge 7 commits into from

Conversation

tlyon3
Copy link
Collaborator

@tlyon3 tlyon3 commented Jul 30, 2020

Most of the data is at the state level. The more granular data is separated into different districts, where a district could have 1 or more counties grouped. I'm not sure how to store this data

@tlyon3 tlyon3 requested review from sglyon and cc7768 July 30, 2020 22:09
@tlyon3
Copy link
Collaborator Author

tlyon3 commented Aug 5, 2020

@cc7768 @sglyon , this is ready. The failure is coming from the FloridaHospitals scraper

Copy link
Contributor

@cc7768 cc7768 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one looked like it was a bit tricky -- Nice work getting it all.

One unfortunate thing is that it doesn't seem like Utah is choosing to report much of the information that we're prioritizing now.

I'd vote to leave this on the back burner for now (and cross our fingers that they choose to make a more useful dashboard) and focus on on our short-term test coverage goals. Thoughts @sglyon @tlyon3 ?

return (
sorts[["dt", "negative_tests_total", "positive_tests_total"]]
.melt(id_vars=["dt"], var_name="variable_name")
.assign(vintage=pd.Timestamp.utcnow(), fips=self.state_fips)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a new method for fetching the vintage so that we are consistent in how we set the vintage. See DatasetBaseNoDate._retrieve_vintage

.agg("sum")
.reset_index()[["dt", "cases_total", "cumulative_hospitalized"]]
.melt(id_vars=["dt"], var_name="variable_name")
.assign(vintage=pd.Timestamp.utcnow(), fips=self.state_fips)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other vintage comment

}
)
renamed["dt"] = (
renamed["dt"].map(lambda x: pd.datetime.fromtimestamp(x / 1000)).dt.date
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the ArcGIS._esri_ts_to_dt method rather than do this by hand.

"DISTNAME": "district",
"COVID_Cases_Total": "cases_total",
"Day": "dt",
"Hospitalizations": "cumulative_hospitalized",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't currently collect cumulative hospitalization data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants