Running Pytest From Repo Master

[TE-Good]

Describe the problem

I installed the latest django-constance from this git repo, through the poetry command:
poetry add --git=https://github.com/jazzband/django-constance.git -v --allow-prereleases django-constance.

My issue is, when running pytest in an activated conda environment I get a the following error;

Traceback (most recent call last):
  File "/Users/tom.good/miniconda3/envs/eigen/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 58, in main
    config = _prepareconfig(args, plugins)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 185, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 653, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 825, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 777, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 96, in load
    module = import_module(match.group('module'))
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/__init__.py", line 2, in <module>
    from . import checks
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 290, in load_module
    six.exec_(co, mod.__dict__)
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/checks.py", line 4, in <module>
    from . import settings
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 290, in load_module
    six.exec_(co, mod.__dict__)
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/settings.py", line 8, in <module>
    'constance.backends.redisd.RedisBackend'
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/django/conf/__init__.py", line 64, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting CONSTANCE_BACKEND, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

To compare, I installed the released 2.6 version, and with our settings configuration, we had no errors. I added this through the command: poetry add django-constance.

When running runserver on both installs, there are no errors. It seems to be isolated to pytest.

Steps to reproduce

See in above description.

System configuration

• Django version: 2.2.13
• Python version: 3.7
• Django-Constance version: 2.6.0 2eb3fce

[Mogost]

Probably problem from #338
@TamoshaytisV could you check?
That may be block #408

[TE-Good]

If you need any more information - let me know!

[TamoshaytisV]

@Mogost Checking your suggestion in #338 (review)

[TamoshaytisV]

@TE-Good can you please tell if you use pytest-django in your project?

[TE-Good]

@TamoshaytisV Yeah, pytest-django version 3.3.3

[TamoshaytisV]

@Mogost i created PR which skips pytest hooks but after some investigation, I think the issue is not with constance pytest plugin. From what I see in the issue description's traceback + source code, that is the normal process of loading django-constance module

/django-constance/constance/checks.py", line 4, in <module>
    from . import settings
/django-constance/constance/settings.py", line 8, in <module>

also in the django sources I see

def _setup(self, name=None):
        """
        Load the settings module pointed to by the environment variable. This
        is used the first time settings are needed, if the user hasn't
        configured settings manually.
        """
        settings_module = os.environ.get(ENVIRONMENT_VARIABLE)
        if not settings_module:
            desc = ("setting %s" % name) if name else "settings"
            raise ImproperlyConfigured(
                "Requested %s, but settings are not configured. "
                "You must either define the environment variable %s "
                "or call settings.configure() before accessing settings."
                % (desc, ENVIRONMENT_VARIABLE))

        self._wrapped = Settings(settings_module)

Which raises error that @TE-Good reported. That exception fires when there's no DJANGO_SETTINGS_MODULE env variable defined. CMIIW

@TE-Good can you please tell how you run pytest and which params you pass to it? Can you please try to set DJANGO_SETTINGS_MODULE env variable and try again or ensure it has the value

[Mogost]

I would invite @blueyed here. Could you help us with this?

[TE-Good]

I reinstalled the package from master, through poetry, ran pytest and pytest -vvx -n 4 --dist=loadscope. Both resulted in the same error again;

django.core.exceptions.ImproperlyConfigured: Requested setting CONSTANCE_BACKEND, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

I set the export DJANGO_SETTINGS_MODULE=eigenapp.settings in my terminal. As that is the name of my core app in django. Ran pytest again and I get the following error;

Traceback (most recent call last):
  File "/Users/tom.good/miniconda3/envs/eigen/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 58, in main
    config = _prepareconfig(args, plugins)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 185, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 653, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 825, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/config/__init__.py", line 777, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 96, in load
    module = import_module(match.group('module'))
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/__init__.py", line 2, in <module>
    from . import checks
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 290, in load_module
    six.exec_(co, mod.__dict__)
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/checks.py", line 4, in <module>
    from . import settings
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 290, in load_module
    six.exec_(co, mod.__dict__)
  File "/Users/tom.good/miniconda3/envs/eigen/src/django-constance/constance/settings.py", line 8, in <module>
    'constance.backends.redisd.RedisBackend'
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/Users/tom.good/miniconda3/envs/eigen/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'eigenapp'

[TamoshaytisV]

@TE-Good Okay, good news :)
So now you need to specify the correct settings module of your app. Can you please play around with it to solve the issue? Or if you will post your project structure, someone can help you with it

If you are using pytest-django, you can just add --ds=myproj.eigenapp.settings parameter of pytest to do quick investigation instead of changing DJANGO_SETTINGS_MODULE all the time.

Any other inputs appreciated here @Mogost @blueyed

[TE-Good]

Thanks @TamoshaytisV! I appreciate yours, and everyone else's time on this.

What is a bit curious to me is if I install django-constance 2.6.0, it all runs well with no errors. I have a settings.py in eigenapp directory. In my manage.py, it also routes to it;

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "eigenapp.settings")
    try:
        from django.core.management import execute_from_command_line
    except ImportError:
        # The above import may fail for some other reason. Ensure that the
        # issue is really that Django is missing to avoid masking other
        # exceptions on Python 2.
        try:
            import django
        except ImportError:
            raise ImportError(
                "Couldn't import Django. Are you sure it's installed and "
                "available on your PYTHONPATH environment variable? Did you "
                "forget to activate a virtual environment?"
            )
        raise
    execute_from_command_line(sys.argv)

I'll try and do a little digging - I'm not very versed in all this though!