High level refactor requirements

[slarse]

This issue contains a list of high-level requirements for refactoring the tests.

1

• Title: Packaging
• Description: The project is currently a mishmash of modules. It needs to be properly packaged for the test suite to be able to collect tests. With the current structure, each top-level directory will be a separate package. Name conflicts could make this unsustainable, so it may be necessary to create subpackages instead.

2

• Title: Add test suite
• Description: Tests are currently spread about the project in various forms, including docstrings, main functions and unit test classes. These should be moved to a separate test suit.

3

• Title: Convert print statements to tests
• Description: Many modules include print statements that seemingly are meant for manual inspection of output. These can be refactored into test cases. The print statements themselves should be left alone.

4

• Title: Ease of access
• Description: The project is for educational purposes, and is likely accessed by people who are not intimately familiar with Python packaging. It is therefore important that the code should be executable without installing anything. In other words, it should be possible to just cd into a directory and run a module.

5

• Title: Fix broken tests
• Description: Any broken tests should be fixed.

6

• Title: Coverage metrics
• Description: Branch+statement coverage metrics should be used.

7

• Title: Continuous integration
• Description: Include all necessary configuration for Travis to run the test suite.

8 (DEPRECATED)

• Deprecation: Initially, we thought the project supported Python 3 and 2. It turns out there is no coherent version support. In early test runs, we found that 150/256 modules could be imported with Python 2.7, while 171/256 could be imported with Python 3.6. The project's own CI also only runs flake8 for Python 3.6. We have therefore decided to deprecate this requirement.
• Title: Backwards compatibility
• Description: The project still maintains compatibility with Python 2.7, and 3.5+

9

• Title: Document test suite
• Description: Instructions on how to run and expand the test suite should be thorough and written for a target audience with little Python experience.

10 (DEPRECATED)

• Deprecation: We thought that the project had no external dependencies, and that the maintainers wanted none. However, we have since found many external dependencies (such as numpy and matplotlib) in the production code, and also had a conversation with a maintainer who seemed positive to using pytest.
• Title: Be as test-runner agnostic as possible
• Description: Most Python test runners can run the standard library unittest tests. As such, tests should be implemented only with standard library features, so the library maintainers can easily swap out the test runner.

11

• Title: Preserve functionality
• Description: Many of the main-functions contain some kind of end-user functionality, such as interactive prompts or just print statements showing the state of a data structure or algorithm. None of this functionality may be touched, but it may be necessary to enclose top level code in functions.