Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Cast non-hashable types to floats in toms748_scan #2467

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

matthewfeickert
Copy link
Member

@matthewfeickert matthewfeickert commented Apr 18, 2024

Description

Resolves #2466

As toms748_scan uses caching of results it requires that the results are hashable objects, which arrays are not. So to keep using this approach convert to floats as needed.

Add testing for all backends for the upper limits inference tests.

Checklist Before Requesting Reviewer

  • Tests are passing
  • "WIP" removed from the title of the pull request
  • Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

  • Summarize commit messages into a comprehensive review of the PR
* In pyhf.infer.intervals.upper_limits.toms748_scan a caching approach to a dict
  is used. This requires that the objects cached are hashable, but generic
  ndarrays are not. To avoid this problem, cast single value tensors to floats
  as needed.
* Use pyhf.tensorlib.transpose to support TensorFlow, as the `.T` operator is not
  supported.
* Add testing of all backends to the toms748_scan and upper_limit tests in 
  tests/test_infer.py.
   - Add notes to change tests after TensorFlow support is dropped.

@matthewfeickert matthewfeickert added tests pytest fix A bug fix need-to-backport tmp label until can be backported to patch release branch labels Apr 18, 2024
@matthewfeickert matthewfeickert self-assigned this Apr 18, 2024
@@ -72,18 +73,20 @@ def test_toms748_scan_bounds_extension(hypotest_args):
data, model, 3, 5, rtol=1e-8
)

assert observed_limit == pytest.approx(observed_limit_default)
# FIXME: Remove float cast after TensorFlow support removed
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kratsg Running the TensorFlow tests is very slow and adds minutes to this run. As we're going to be dropping TensorFlow in the future, I think that we could probably just skip TensorFlow in these tests. Thoughts?

@matthewfeickert matthewfeickert requested a review from kratsg April 18, 2024 06:53
@matthewfeickert
Copy link
Member Author

CI is currently failing for Apple silicon macOS.

Copy link

codecov bot commented Apr 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.21%. Comparing base (64ab264) to head (e815612).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2467   +/-   ##
=======================================
  Coverage   98.21%   98.21%           
=======================================
  Files          69       69           
  Lines        4543     4546    +3     
  Branches      804      805    +1     
=======================================
+ Hits         4462     4465    +3     
  Misses         48       48           
  Partials       33       33           
Flag Coverage Δ
contrib 97.80% <100.00%> (+<0.01%) ⬆️
doctest 98.08% <100.00%> (+<0.01%) ⬆️
unittests-3.10 ?
unittests-3.11 ?
unittests-3.12 96.23% <100.00%> (+<0.01%) ⬆️
unittests-3.8 96.26% <100.00%> (+<0.01%) ⬆️
unittests-3.9 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -15,7 +15,8 @@ def __dir__():

def _interp(x, xp, fp):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _interp(x, xp, fp):
def _interp(x, xp: list[float], fp):

@@ -15,7 +15,8 @@ def __dir__():

def _interp(x, xp, fp):
tb, _ = get_backend()
return tb.astensor(np.interp(x, xp.tolist(), fp.tolist()))
# xp has already been turned into a list at this point
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can drop this comment with a typehint above as suggested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix A bug fix need-to-backport tmp label until can be backported to patch release branch tests pytest
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

toms748_scan doesn't work with JAX backend
2 participants