-
Notifications
You must be signed in to change notification settings - Fork 13
expects decorator #143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
TomNicholas
wants to merge
55
commits into
xarray-contrib:main
Choose a base branch
from
TomNicholas:expects_decorator
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
expects decorator #143
Changes from all commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
044d59a
draft implementation of @expects
TomNicholas 0754f22
sketch of different tests needed
TomNicholas e879ef9
idea for test
TomNicholas aad7936
upgrade check then convert function to optionally take magnitude
TomNicholas e354f4e
removed magnitude option
TomNicholas 7727d8e
works for single return value
TomNicholas 1379779
works for single kwarg
TomNicholas 77f5d02
works for multiple return values
TomNicholas 71f4200
allow passing through arguments unchecked
TomNicholas a710741
check types of units
TomNicholas 497e97f
remove uneeded option to specify a lack of return value
TomNicholas 00219bc
check number of inputs and return values
TomNicholas 9e92f21
removed nonlocal keyword
TomNicholas 86f7e58
generalised to handle specifying dicts of units
TomNicholas 2141c6c
type hint for func
TomNicholas a94a6ae
type hint for args_units
TomNicholas a2cc63f
Merge branch 'expects_decorator' of https://github.com/TomNicholas/pi…
TomNicholas 7103483
numpy-style type hints for all arguments
TomNicholas 59ddf86
whats new
TomNicholas a5a2493
add to API docs
TomNicholas e5e84fb
use always_iterable
TomNicholas 3f59414
hashable
TomNicholas b281674
hashable
TomNicholas c669105
hashable
TomNicholas 9ac8887
dict comprehension
TomNicholas 0a6447d
list comprehension
TomNicholas c29f935
unindent if/else
TomNicholas 81913a6
missing parenthesis
TomNicholas 4de6f4d
simplify if/else logic for checking there were actually results
TomNicholas 83e422f
return results immediately if a tuple
TomNicholas 37c3fbc
allow for returning Datasets from wrapped funciton
TomNicholas 9c19af0
Update docs/api.rst
TomNicholas 0b5c7c0
correct indentation of docstring
TomNicholas 0f50305
use inspects to check number of arguments passed to decorated function
TomNicholas 57d341e
reformat the docstring
keewis 8845b77
update the definition of unit-like
keewis bc41425
simplify if/else statement
TomNicholas aba2d11
Merge branch 'expects_decorator' of https://github.com/TomNicholas/pi…
TomNicholas 0350308
check units in .to instead
TomNicholas 3a24a73
remove extra xfailed test
TomNicholas 19fd6e0
test raises on unquantified input
TomNicholas d2d74e4
add example of function which optionally accepts dimensionless weights
TomNicholas 1c4feb4
Merge branch 'main' into expects_decorator
keewis 7a6f2cb
Merge branch 'main' into expects_decorator
keewis 85b982c
rewrite using inspect.Signature's bind and bind_partial
keewis 5ea484b
also allow converting and stripping Variable objects
keewis b7c71c1
implement the conversion functions
keewis b39fff3
simplify the return construct
keewis 61e0299
code reorganization
keewis 63d8aeb
black
keewis 32a57b2
fix a test
keewis 91b4826
remove the note about coordinates not being checked [skip-ci]
keewis a43dd13
reword the error message raised when there's no units for some parame…
keewis 7ea921c
move the changelog to a new section
keewis b92087a
Merge branch 'main' into expects_decorator
keewis File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,255 @@ | ||
import functools | ||
import inspect | ||
from inspect import Parameter | ||
|
||
from pint import Quantity | ||
from xarray import DataArray, Dataset, Variable | ||
|
||
from . import conversion | ||
from .accessors import PintDataArrayAccessor # noqa | ||
|
||
|
||
def detect_missing_params(params, units): | ||
"""detect parameters for which no units were specified""" | ||
variable_params = { | ||
Parameter.VAR_POSITIONAL, | ||
Parameter.VAR_KEYWORD, | ||
} | ||
|
||
return { | ||
name | ||
for name, param in params.items() | ||
if name not in units.arguments and param.kind not in variable_params | ||
} | ||
|
||
|
||
def convert_and_strip(obj, units): | ||
if isinstance(obj, (DataArray, Dataset, Variable)): | ||
if not isinstance(units, dict): | ||
units = {None: units} | ||
return conversion.strip_units(conversion.convert_units(obj, units)) | ||
elif isinstance(obj, Quantity): | ||
return obj.m_as(units) | ||
elif units is None: | ||
return obj | ||
else: | ||
raise ValueError(f"unknown type: {type(obj)}") | ||
|
||
|
||
def convert_and_strip_args(args, units): | ||
return [convert_and_strip(obj, units_) for obj, units_ in zip(args, units)] | ||
|
||
|
||
def convert_and_strip_kwargs(kwargs, units): | ||
return {name: convert_and_strip(kwargs[name], units[name]) for name in kwargs} | ||
|
||
|
||
def always_iterable(obj, base_type=(str, bytes)): | ||
""" | ||
If *obj* is iterable, return an iterator over its items, | ||
If *obj* is not iterable, return a one-item iterable containing *obj*, | ||
If *obj* is ``None``, return an empty iterable. | ||
If *base_type* is set, objects for which ``isinstance(obj, base_type)`` | ||
returns ``True`` won't be considered iterable. | ||
|
||
Copied from more_itertools. | ||
""" | ||
|
||
if obj is None: | ||
return iter(()) | ||
|
||
if (base_type is not None) and isinstance(obj, base_type): | ||
return iter((obj,)) | ||
|
||
try: | ||
return iter(obj) | ||
except TypeError: | ||
return iter((obj,)) | ||
|
||
|
||
def attach_return_units(results, units): | ||
if units is None: | ||
# ignore types and units of return values | ||
return results | ||
elif results is None: | ||
raise TypeError( | ||
"Expected function to return something, but function returned None" | ||
) | ||
else: | ||
# handle case of function returning only one result by promoting to 1-element tuple | ||
return_units_iterable = tuple(always_iterable(units, base_type=(str, dict))) | ||
results_iterable = tuple(always_iterable(results, base_type=(str, Dataset))) | ||
|
||
# check same number of things were returned as expected | ||
if len(results_iterable) != len(return_units_iterable): | ||
raise TypeError( | ||
f"{len(results_iterable)} return values were received, but {len(return_units_iterable)} " | ||
"return values were expected" | ||
) | ||
|
||
converted_results = _attach_multiple_units( | ||
results_iterable, return_units_iterable | ||
) | ||
|
||
if isinstance(results, tuple) or len(converted_results) != 1: | ||
return converted_results | ||
else: | ||
return converted_results[0] | ||
|
||
|
||
def _check_or_convert_to_then_strip(obj, units): | ||
""" | ||
Checks the object is of a valid type (Quantity or DataArray), then attempts to convert it to the specified units, | ||
then strips the units from it. | ||
""" | ||
|
||
if units is None: | ||
# allow for passing through non-numerical arguments | ||
return obj | ||
elif isinstance(obj, Quantity): | ||
converted = obj.to(units) | ||
return converted.magnitude | ||
elif isinstance(obj, (DataArray, Dataset)): | ||
converted = obj.pint.to(units) | ||
return converted.pint.dequantify() | ||
else: | ||
raise TypeError( | ||
"Can only expect units for arguments of type xarray.DataArray," | ||
f" xarray.Dataset, or pint.Quantity, not {type(obj)}" | ||
) | ||
|
||
|
||
def _attach_units(obj, units): | ||
"""Attaches units, but can also create pint.Quantity objects from numpy scalars""" | ||
if isinstance(obj, (DataArray, Dataset)): | ||
return obj.pint.quantify(units) | ||
else: | ||
return Quantity(obj, units=units) | ||
|
||
|
||
def _attach_multiple_units(objects, units): | ||
"""Attaches list of units to list of objects elementwise""" | ||
converted_objects = [_attach_units(obj, unit) for obj, unit in zip(objects, units)] | ||
return converted_objects | ||
|
||
|
||
def expects(*args_units, return_units=None, **kwargs_units): | ||
""" | ||
Decorator which ensures the inputs and outputs of the decorated | ||
function are expressed in the expected units. | ||
|
||
Arguments to the decorated function are checked for the specified | ||
units, converting to those units if necessary, and then stripped | ||
of their units before being passed into the undecorated | ||
function. Therefore the undecorated function should expect | ||
unquantified DataArrays, Datasets, or numpy-like arrays, but with | ||
the values expressed in specific units. | ||
|
||
Parameters | ||
---------- | ||
func : callable | ||
Function to decorate, which accepts zero or more | ||
xarray.DataArrays or numpy-like arrays as inputs, and may | ||
optionally return one or more xarray.DataArrays or numpy-like | ||
arrays. | ||
*args_units : unit-like or mapping of hashable to unit-like, optional | ||
Units to expect for each positional argument given to func. | ||
|
||
The decorator will first check that arguments passed to the | ||
decorated function possess these specific units (or will | ||
attempt to convert the argument to these units), then will | ||
strip the units before passing the magnitude to the wrapped | ||
function. | ||
|
||
A value of None indicates not to check that argument for units | ||
(suitable for flags and other non-data arguments). | ||
return_units : unit-like or list of unit-like or mapping of hashable to unit-like \ | ||
or list of mapping of hashable to unit-like, optional | ||
The expected units of the returned value(s), either as a | ||
single unit or as a list of units. The decorator will attach | ||
these units to the variables returned from the function. | ||
|
||
A value of None indicates not to attach any units to that | ||
return value (suitable for flags and other non-data results). | ||
keewis marked this conversation as resolved.
Show resolved
Hide resolved
|
||
kwargs_units : mapping of hashable to unit-like, optional | ||
Unit to expect for each keyword argument given to func. | ||
|
||
The decorator will first check that arguments passed to the | ||
decorated function possess these specific units (or will | ||
attempt to convert the argument to these units), then will | ||
strip the units before passing the magnitude to the wrapped | ||
function. | ||
|
||
A value of None indicates not to check that argument for units | ||
(suitable for flags and other non-data arguments). | ||
keewis marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Returns | ||
------- | ||
return_values : Any | ||
Return values of the wrapped function, either a single value | ||
or a tuple of values. These will be given units according to | ||
return_units. | ||
|
||
Raises | ||
------ | ||
TypeError | ||
If an argument or return value has a specified unit, but is | ||
not an xarray.DataArray or pint.Quantity. Also thrown if any | ||
of the units are not a valid type, or if the number of | ||
arguments or return values does not match the number of units | ||
specified. | ||
Comment on lines
+196
to
+200
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. shouldn't we raise There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah possibly. I can change that. |
||
|
||
Examples | ||
-------- | ||
|
||
Decorating a function which takes one quantified input, but | ||
returns a non-data value (in this case a boolean). | ||
|
||
>>> @expects("deg C") | ||
... def above_freezing(temp): | ||
... return temp > 0 | ||
|
||
Decorating a function which allows any dimensions for the array, but also | ||
accepts an optional `weights` keyword argument, which must be dimensionless. | ||
|
||
>>> @expects(None, weights="dimensionless") | ||
... def mean(da, weights=None): | ||
... if weights: | ||
... return da.weighted(weights=weights).mean() | ||
... else: | ||
... return da.mean() | ||
|
||
""" | ||
|
||
def _expects_decorator(func): | ||
|
||
# check same number of arguments were passed as expected | ||
sig = inspect.signature(func) | ||
|
||
params = sig.parameters | ||
|
||
bound_units = sig.bind_partial(*args_units, **kwargs_units) | ||
|
||
missing_params = detect_missing_params(params, bound_units) | ||
if missing_params: | ||
raise TypeError( | ||
"Some parameters of the decorated function are missing units:" | ||
f" {', '.join(sorted(missing_params))}" | ||
) | ||
|
||
@functools.wraps(func) | ||
def _unit_checking_wrapper(*args, **kwargs): | ||
bound = sig.bind(*args, **kwargs) | ||
|
||
converted_args = convert_and_strip_args(bound.args, bound_units.args) | ||
converted_kwargs = convert_and_strip_kwargs( | ||
bound.kwargs, bound_units.kwargs | ||
) | ||
|
||
results = func(*converted_args, **converted_kwargs) | ||
|
||
return attach_return_units(results, return_units) | ||
|
||
return _unit_checking_wrapper | ||
|
||
return _expects_decorator |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.