I think that this bug was meant to be about scipy 1.14.1 and 1.15.1/1.15.2 rather than 1.4.1 and 1.5.1/1.5.2.

It looks like 1.15.2 was released on 2025-02-16, so it might be possible to proceed with this. I’m not able to reproduce issues with from scipy.stats import levene using scipy==1.15.1 in a virtualenv, so I can’t confirm whether or not scipy==1.15.2 fixes them.

#187 is still red. Not sure what the reason is, and indeed I run with scipy==1.5.2 locally with no problem. So at a guess it's something about the CI setup. Did not have time to figure it out. If you've ideas, they're greatly appreciated.

It’s an odd issue, indeed.

scipy/scipy#22257 (comment) seems suspiciously similar, and I asked the reporter if they have any more information.

I am able to reproduce it on main as follows:

$ python3.11 -m venv _e
$ . _e/bin/activate
(_e) $ pip install -e .[test]
(_e) $ $ pip install --upgrade scipy
(_e) $ python -m pytest --pyargs scipy_doctest
[…]
 scipy_doctest.tests.module_cases.func8
 --------------------------------------

File "/home/ben/src/forks/scipy_doctest/scipy_doctest/tests/module_cases.py", line 114, in scipy_doct
est.tests.module_cases.func8
Failed example:
    from scipy.stats import levene
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib64/python3.11/doctest.py", line 1355, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest scipy_doctest.tests.module_cases.func8[0]>", line 1, in <module>
        from scipy.stats import levene
      File "/home/ben/src/forks/scipy_doctest/_e/lib64/python3.11/site-packages/scipy/stats/__init__.py", line 624, in <module>
        from ._stats_py import *
      File "/home/ben/src/forks/scipy_doctest/_e/lib64/python3.11/site-packages/scipy/stats/_stats_py.py", line 41, in <module>
        from scipy.optimize import milp, LinearConstraint
      File "/home/ben/src/forks/scipy_doctest/_e/lib64/python3.11/site-packages/scipy/optimize/__init__.py", line 435, in <module>
        from ._linprog import linprog, linprog_verbose_callback
      File "/home/ben/src/forks/scipy_doctest/_e/lib64/python3.11/site-packages/scipy/optimize/_linprog.py", line 21, in <module>
        from ._linprog_highs import _linprog_highs
      File "/home/ben/src/forks/scipy_doctest/_e/lib64/python3.11/site-packages/scipy/optimize/_linprog_highs.py", line 27, in <module>
        from ._highspy._core.simplex_constants import (
    ModuleNotFoundError: No module named 'scipy.optimize._highspy._core.simplex_constants'; 'scipy.optimize._highspy._core' is not a package
Trying:
    a = [8.88, 9.12, 9.04, 8.98, 9.00, 9.08, 9.01, 8.85, 9.06, 8.99]
Expecting nothing
[…]
====================================== short test summary info ======================================
FAILED scipy_doctest/tests/test_testmod.py::test_module - assert 4 == 0
FAILED scipy_doctest/tests/test_testmod.py::test_module_vanilla_dtfinder - assert 4 == 0
FAILED scipy_doctest/tests/test_testmod.py::test_public_obj_discovery - assert 4 == 0
============================== 3 failed, 56 passed, 1 xfailed in 1.02s ==============================

I started with Python 3.11 because that’s what the CI was trying when it failed, but I can reproduce it with 3.12 and 3.13, too.

https://github.com/scipy/scipy/blob/main/scipy/optimize/_highspy/meson.build#L65-L72 generates the apparently problematic module

https://github.com/scipy/scipy/blob/main/scipy/optimize/_highspy/meson.build#L65-L72 generates the apparently problematic module

It’s not quite the same issue as the original report scipy/scipy#22257, but there is still something messy going on with global state. Simply running python3 -c 'from scipy.stats import levene' in the environment described above works fine.

Perhaps we can make the error go away by tweaking Pytest assertion rewriting? That's my most recent memory of a bug only reproducible via pytest

Okay, I can now repro under python 3.13. Am pretty sure it did not error under self-built scipy ($ python dev.py build && python dev.py shell) on earlier pythons.
Note that it just works in the SciPy CI, also on an older python.
Which is probably why we did not see it before.

To make it more bizzare:

1. $ pytest scipy_doctest/tests -v fails in test_testmod.py executing 'scipy_doctest.tests.module_cases.func8'
2. executing $ pytest scipy_doctest/tests/test_testmod.py -v does not fail.

When it fails, dropping a breakpoint shows something really weird:

(Pdb) from scipy.stats import levene
*** ModuleNotFoundError: No module named 'scipy.optimize._highspy._core.simplex_constants'; 'scipy.optimize._highspy._core' is not a package
(Pdb) import scipy.stats as stats
*** ModuleNotFoundError: No module named 'scipy.optimize._highspy._core.simplex_constants'; 'scipy.optimize._highspy._core' is not a package
(Pdb) import scipy
(Pdb) scipy.stats.levene
<function levene at 0x7f4203b6dbc0>
(Pdb) p test.name
'scipy_doctest.tests.module_cases.func8'

Re assertion rewrites: #166 did have to work around it. Running

$ pytest scipy_doctest/tests/ --assert=plain

shows the same error.

how bizarre!

1. $ pytest scipy_doctest/tests -v fails in test_testmod.py executing 'scipy_doctest.tests.module_cases.func8'
2. executing $ pytest scipy_doctest/tests/test_testmod.py -v does not fail.

I feel like this warrants a pytest issue, no? Not that a MRE will likely be easy

More breadcrumbs:

$ git diff
diff --git a/scipy_doctest/frontend.py b/scipy_doctest/frontend.py
index 165d5d4..fbe4b19 100644
--- a/scipy_doctest/frontend.py
+++ b/scipy_doctest/frontend.py

diff --git a/scipy_doctest/tests/test_testmod.py b/scipy_doctest/tests/test_testmod.py
index 250b2e9..783570a 100644
--- a/scipy_doctest/tests/test_testmod.py
+++ b/scipy_doctest/tests/test_testmod.py
@@ -19,6 +19,8 @@ try:
 except Exception:
     HAVE_SCIPY = False
 
+from scipy.stats import levene
+
 from . import (module_cases as module,
                stopwords_cases as stopwords,
                finder_cases,

makes the error go away.

Trying it without scipy-doctest or with it but outside of the test suite: all seems good.

$ cat mwe_scipy115.py 
def func():
    """
    from scipy.stats import levene

    >>> 2 + 3
    5
    """
    pass
br@gonzales:~/sweethome/temp$ pytest --doctest-modules mwe_scipy115.py 
================================= test session starts ==================================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/br/sweethome/temp
plugins: json-report-1.5.0, hypothesis-6.125.1, xdist-3.6.1, scipy_doctest-1.8.dev0, anyio-4.8.0, cov-6.0.0, timeout-2.3.1, doctestplus-1.4.0, metadata-3.1.1
collected 1 item                                                                       

mwe_scipy115.py .                                                                [100%]

================================== 1 passed in 0.52s ===================================

So the error is somewhere in how the test suite is organized. I think the key is to understand the difference between what we do locally and what goes on the scipy CI: https://github.com/scipy/scipy/blob/75452588839d75a562c77c014ac027c3a7ed4ee7/.circleci/config.yml#L187

Now, this reminds me of #131 (comment)
which itself points to matplotlib/matplotlib#26827 and gist is that import matplotlib.pyplot did work while import matplotlib caused mysterious errors.

So I added the workaround to #187. Will try bisecting as a last ditch attempt at figuring if it's scipy-doctest. If that does not find anything, yeah, probably an upstream issue is called for.

No dice: going all the way down to scipy-doctest==1.1 still error out unless there's import scipy.stats at global scope.
Downgrading to pytest down to pytest==7 or pytest==8.2 still errors out.
Changing pytest --import-mode=X for X in (importlib, prepend, append) still errors out, so it's not pytest's import mode.
So the problem is somewhere in the combination of scipy==1.5.x and whatever scipy-doctest is doing.
I formally declare being at my wits end.
There's a workaround in gh-184: import scipy.stats at global scope.
Why? I've no idea, not any better than gh-131: "something is weird in the way the top-level scipy namespace is organized".
So an upstream issue would be great @lucascolley , if you're up to it.

Well, I'm not really sure what to report 😅 I doubt they will be very amenable to just "here's our whole test suite, why does it pass with this command but fail with this other one"

Okay, so

1. we have a workaround in MAINT : stop upper capping scipy #187 : just import scipy at the global scope

2. The problem is gone if I manually comment out one test (not failing one, no): https://github.com/scipy/scipy_doctest/blob/main/scipy_doctest/tests/test_pytest_configuration.py#L26

I asked the best I could distill at hopefully the right forum: pytest-dev/pytest#13353
If there's help from pytest core, great. If not, we goto 1.

On a bright side @musicinmybrain , the problem is definitely within the scipy-doctest testing machinery. It might take a while to sort out, but whatever the fix, it should not block your packaging work, I'd think.

On a bright side @musicinmybrain , the problem is definitely within the scipy-doctest testing machinery. It might take a while to sort out, but whatever the fix, it should not block your packaging work, I'd think.

Thanks. I’ll monitor this issue, and if Fedora’s scipy package is updated to 1.15.x before you find a proper resolution, the workaround in #187 should suffice as a patch.

Reopen to keep track of an upstream issue, pytest-dev/pytest#13353
The workaround is in, so we're good meanwhile.