Extend set_meta_from_data() to apply on different column - #986
Extend set_meta_from_data() to apply on different column#986danielhuppmann wants to merge 7 commits into
set_meta_from_data() to apply on different column#986Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #986 +/- ##
=======================================
- Coverage 94.7% 94.7% -0.1%
=======================================
Files 70 70
Lines 6717 6732 +15
=======================================
+ Hits 6365 6379 +14
- Misses 352 353 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
phackstock
left a comment
There was a problem hiding this comment.
Looks good to me. Just a few minor comments and questions below.
On the topic of using on for the name of the column that is used to populate the values in the meta table, I think using on is good. I always like taking design patterns, or variable names in this case, from other, established libraries.
| value = x[x[on] == x[on].apply(method)][column].unique() | ||
|
|
||
| if len(value) > 1: | ||
| logger.warning(f"Non-unique result from {method} on column {on}.") |
There was a problem hiding this comment.
Should this just be a warning? I'd imagine that producing a non-unique value for a meta value can only be ill-conceived. We might want to raise an actual error here.
There was a problem hiding this comment.
In my original use case, it could be that the temperature stabilises at (for example) 1.7°,, in particular if we round the output from the climate assessment. So there could be multiple years where peak temperature is reached, without this being necessarily incorrect.
The truly correct solution would be to also provide an on_method argument that makes this unique?
Co-authored-by: Philip Hackstock <20710924+phackstock@users.noreply.github.com>
Please confirm that this PR has done the following:
Name of contributors Added to AUTHORS.rstDescription of PR
This PR (finally) implements a feature that a user can check where some function applied to a timeseries in
dataholds and use the corresponding value in a different column for the meta value.I've done this (manually) multiple times for getting the year where peak-warming (i.e., maximum of the temperature timeseries) is reached.
A few items to discuss:
groupby(by)... But I'm open to suggestions for alternative argument namesapply_method()is very "hands-on", there may be more elegant approaches with lumpy...?df.groupby()[column].apply("min")yields a different result fromdf.groupby()[column].apply(np.min)(because the latter evaluates element-wise).Example
To set the year where peak-temperature is reached as meta-indicator "Year of Peak Warming":