Make the test suite independent of installed plugins

[ribbal]

While developing the example redlinks plugin, I noticed that installing it into the development environment changes the behavior of the core test suite.

For example, after installing the plugin with:

pip install -e docs/plugin_examples/plugin_redlinks/

the renderer test

def test_wiki_link():
    text = "[[Title|Link]]"
    html, _, _ = render.markdown(text)
    assert '<a href="/Link">Title</a>' in html

fails because the plugin decorates the generated link by adding attributes such as class="redlink" and inline styles. For example:

<a class="redlink" 
    href="/Link" 
    style="color: #d33; text-decoration: none; border-bottom: 1px dotted #d33;"> Title </a>

The plugin is behaving as intended, but it means the outcome of the core test suite depends on which pluggy plugins are installed in the current Python environment.

Suggestion

It would be useful if the core test suite could disable pluggy plugin loading, or otherwise create an application/plugin manager that doesn't discover external plugins.

This would make the core tests deterministic and independent of the developer's environment. Plugin-specific tests could then explicitly register the plugins they need and verify their behavior in isolation.

Is there an existing mechanism to prevent plugin discovery during tests, or would it make sense to add one?

[redimp]

Hey @ribbal, I understand the issue that you are running into. I suggest using a clean virtual environment for testing, or more simple: Use make tox which sets up a independent environment for running the tests.

[redimp]

Especially since we added tests for the example plugins ;) https://github.com/redimp/otterwiki/tree/main/docs/plugin_examples/tests