feat: handle union type and add pre-commit lint hooks (#51)

* ci: add pre-commit hooks and automated tests

* build: remove obsolete init.sh file

* ci: define a cache for the virtual environment based on the dependencies lock file

* build(yapf): upgraded version

* feat(inspect): handle unions in type annotations

* build: improve code coverage (added tests, removed irrelevant code)

* ci: run test without installing the lint dependencies

* docs(readme): fix copy-pasted reference to pydoctrace

* docs(contributing): add how to activate the pre-commit hooks

* ci(ruff): update version to 0.0.285

* ci(jobs): add concurrency to cancel a running job when a new one is triggered

* refactor(tests): split union-types tests in different unit tests for readability sake

* style(tests): post-review fixes about test code (type annotations, code comments, doc styling, default values)

* docs(readme): add a section about the Python versions required by py2puml and pyenv

Author: lucsorel

.github/workflows/python-package.yml

@@ -0,0 +1,39 @@
+name: Python package
+
+on: [push]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  MIN_CODE_COVERAGE_PERCENT: 93
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Install Python
+        uses: actions/setup-python@v4
+        with:
+          python-version-file: '.python-version'
+
+      - name: Install poetry
+        uses: abatilo/actions-poetry@v2
+        with:
+          poetry-version: 1.5.1
+
+      - name: Define a cache for the virtual environment based on the dependencies lock file
+        uses: actions/cache@v3
+        with:
+          path: ./.venv
+          key: venv-${{ hashFiles('poetry.lock') }}
+
+      - name: Install project dependencies
+        run: poetry install --without lint
+
+      - name: Run automated tests (with code coverage)
+        run: poetry run pytest -v --cov=py2puml --cov-branch --cov-report term-missing --cov-fail-under $MIN_CODE_COVERAGE_PERCENT

.pre-commit-config.yaml

@@ -0,0 +1,45 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+    -   id: check-yaml
+    -   id: check-toml
+    # trims all whitespace from the end of each line
+    -   id: trailing-whitespace
+    # ensures that all files end in a newline and only a newline
+    -   id: end-of-file-fixer
+    # replace "double quotes" by 'single quotes' unless "it's impossible"
+    -   id: double-quote-string-fixer
+    # prevents large files from being committed (>100kb)
+    -   id: check-added-large-files
+        args: [--maxkb=100]
+    # enforce the naming conventions of test scripts
+    -   id: name-tests-test
+        # tests match test_.*\.py
+        args: [--pytest-test-first]
+        exclude: tests.asserts.*|tests.modules.*|tests/py2puml/parsing/mockedinstance.py
+
+-   repo: https://github.com/alessandrojcm/commitlint-pre-commit-hook
+    rev: v9.5.0
+    hooks:
+    -   id: commitlint
+        stages: [commit-msg]
+        additional_dependencies: ['@commitlint/config-angular']
+
+-   repo: https://github.com/PyCQA/isort
+    rev: 5.12.0
+    hooks:
+    -   id: isort
+        additional_dependencies: [toml]
+
+-   repo: https://github.com/google/yapf
+    rev: v0.40.0
+    hooks:
+    -   id: yapf
+        name: Yapf
+        additional_dependencies: [toml]
+
+-   repo: https://github.com/charliermarsh/ruff-pre-commit
+    rev: v0.0.285
+    hooks:
+    -   id: ruff

.python-version

@@ -1 +1 @@
-3.8.6
+3.10.9

CONTRIBUTING.md

@@ -64,6 +64,19 @@ version = "major.minor.patch"
 It takes time to write comprehensive guidelines.
 To save time (my writing time and your reading time), I tried to make it short so my best advice is _go have a look at the production and test codes and try to follow the conventions you draw from what you see_ 🙂
 
+To homogenize code style consistency and enforce code quality, this project uses:
+
+- the `yapf` formatter and the `ruff` linter
+- `pre-commit` hooks that are triggered on the Github CI (in pull-requests) and should be activated locally when contributing to the project:
+
+```sh
+# installs the dependencies, including pre-commit as a lint dependency
+poetry install
+
+# activates the pre-commit hooks
+poetry run pre-commit install --hook-type pre-commit --hook-type commit-msg
+```
+
 ## Unit tests
 
 Pull requests must come with unit tests, either new ones (for feature addtitions), changed ones (for feature changes) or non-regression ones (for bug fixes).
@@ -91,82 +104,3 @@ When manipulating **literal strings**:
 python_version = '3.8+'
 f'use f-strings to format strings, py2puml use Python {python_version}'
 ```
-
-### Code formatting
-
-#### Imports
-
-The following guidelines regarding imports are designed to help people understand what a Python module does or deals with:
-
-* place import statements at the top of the file
-* explicit the functionalities you import with the `from ... import ...` syntax instead of importing a whole module (it does not tell how this module is being used)
-* order the imports by family of modules:
-  1. the native modules (the types imported from the typing module are often placed first)
-  1. the dependency modules defined in the [pyproject.toml file](pyproject.toml) (`py2puml` only use development dependencies)
-  1. the modules of the `py2puml` package
-* when they are so many items imported from a module that they do not fit on a single line, wrap them with a pair of parentheses instead of using a backslash
-
-```python
-from typing import Dict, List, Tuple
-
-from ast import (
-    NodeVisitor, arg, expr,
-    FunctionDef, Assign, AnnAssign,
-    Attribute, Name, Tuple as AstTuple,
-    Subscript, get_source_segment
-)
-from collections import namedtuple
-
-from py2puml.domain.umlclass import UmlAttribute
-from py2puml.domain.umlrelation import UmlRelation, RelType
-```
-
-#### Code indentation
-
-Most settings in Python-code editors replace tabulation by 4 space characters.
-That is what is used for this library as well.
-
-This library follows an opiniated way for formatting the code between **parentheses, brackets or braces**.
-Whenever a block of python expressions is surrounded by a pair of parentheses, brackets or braces:
-
-* start the inner content in an indented block (one tabulation is enough) in a new line following the opening parenthesis, bracket or brace
-* type the closing parenthesis, bracket or brace in a new line, de-indent so that the inner content is visually enclosed between the opening and closing symbol
-* repeat the logic within the inner content block
-
-Some examples:
-
-```python
-# function signature and return type
-def parse_class_constructor(
-    class_type: Type,
-    class_fqn: str, root_module_name: str
-) -> Tuple[
-    List[UmlAttribute], Dict[str, UmlRelation]
-]:
-    constructor = getattr(class_type, '__init__', None)
-    ...
-
-# function call
-print(''.join(
-    py2puml('py2puml/domain', 'py2puml.domain')
-))
-
-# using a generator to get the first matching item
-definition_module_member = next((
-    member for member in definition_members
-    # ensures that the type belongs to the module being parsed
-    if member[0] == '__module__' and member[1].startswith(root_module_name)
-), None)
-
-# dictionary comprehension with a complex filtering expression
-def extend_relations(self, target_fqns: List[str]):
-    self.uml_relations_by_target_fqn.update({
-        target_fqn: UmlRelation(self.class_fqn, target_fqn, RelType.COMPOSITION)
-        for target_fqn in target_fqns
-        if target_fqn.startswith(self.root_fqn) and (
-            target_fqn not in self.uml_relations_by_target_fqn
-        )
-    })
-```
-
-I am looking forward to providing the linting settings corresponding to these practices.

README.md

@@ -11,12 +11,28 @@
 
 Generate PlantUML class diagrams to document your Python application.
 
+[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/lucsorel/py2puml/main.svg)](https://results.pre-commit.ci/latest/github/lucsorel/py2puml/main)
+
+
+`py2puml` uses [pre-commit hooks](https://pre-commit.com/) and [pre-commit.ci Continuous Integration](https://pre-commit.ci/) to enforce commit messages, code formatting and linting for quality and consistency sake.
+See the [code conventions](#code-conventions) section if you would like to contribute to the project.
+
+
 # How it works
 
 `py2puml` produces a class diagram [PlantUML script](https://plantuml.com/en/class-diagram) representing classes properties (static and instance attributes) and their relations (composition and inheritance relationships).
 
 `py2puml` internally uses code [inspection](https://docs.python.org/3/library/inspect.html) (also called *reflexion* in other programming languages) and [abstract tree parsing](https://docs.python.org/3/library/ast.html) to retrieve relevant information.
-Some parsing features are available only since Python 3.8 (like [ast.get_source_segment](https://docs.python.org/3/library/ast.html#ast.get_source_segment)).
+
+## Minimum Python versions to run py2puml
+
+`p2puml` uses some code-parsing features that are available only since **Python 3.8** (like [ast.get_source_segment](https://docs.python.org/3/library/ast.html#ast.get_source_segment)).
+If your codebase uses the `int | float` syntax to define optional types, then you should use Python 3.10 to run `py2puml`.
+
+To sum it up, use at least Python 3.8 to run py2puml, or a higher version if you use syntax features available only in higher versions.
+
+The [.python-version](.python-version) file indicates the Python version used to develop the library.
+It is a file used by [pyenv](https://github.com/pyenv/pyenv/) to define the binary used by the project.
 
 ## Features
 
@@ -129,7 +145,7 @@ footer Generated by //py2puml//
 @enduml
 ```
 
-Using PlantUML, this content is rendered as this diagram:
+Using PlantUML, this content is rendered as in this diagram:
 
 ![py2puml domain UML Diagram](https://www.plantuml.com/plantuml/png/ZPD1Yzim48Nl-XLpNbWRUZHxs2M4rj1DbZGzbIN8zcmgAikkD2wO9F-zigqWEw1L3i6HPgJlFUdfsH3NrDKIslvBQxz9rTHSAAPuZQRb9TuKuCG0PaLU_k5776S1IicDkLcGk9RaRT4wRPA18Ut6vMyXAuqgW-_2q2_N_kwgWh0s1zNL1UeCXA9n_iAcdnTamQEApnHTUvAVjNmXqgBeAAoB-dOnDiH9b1aKJIETYBj8gvai07xb6kTtfiMRDWTUM38loV62feVpYNWUMWOXkVq6tNxyLMuO8g7g8gIn9Nd5uQw2e7zSTZX7HJUqqjUU3L2FWElvJRZti6wDafDeb5i_shWb-QvaXtBVjpuMg-ths_P7li-tcmmUu3J5uEAg-URRUfVlNpQhTGPFPr-EUlD4ws-tr0XWcawNU5ZS2W1nVKJoi_EWEjspSxYmo8jyU7oCF5eMoxNV8_BCM2INJsUxKOp68WdnOWAfl5j56CBkl4cd9H8pzj4qX1g-eaBD2IieUaXJjp1DsJEgolvZ_m40)
 
@@ -160,9 +176,9 @@ docker start plantumlserver
 
 ## Python API
 
-For example, to create the diagram of the classes used by `py2puml`:
+For example, to create the diagram of the domain classes used by `py2puml`:
 
-* import the `py2puml` function in your script (see [py2puml/example.py](py2puml/example.py)):
+* import the `py2puml` function in your script:
 
 ```python
 from py2puml.py2puml import py2puml
@@ -172,11 +188,11 @@ if __name__ == '__main__':
     print(''.join(py2puml('py2puml/domain', 'py2puml.domain')))
 
     # writes the PlantUML content in a file
-    with open('py2puml/domain.puml', 'w') as puml_file:
+    with open('py2puml/py2puml.domain.puml', 'w', encoding='utf8') as puml_file:
         puml_file.writelines(py2puml('py2puml/domain', 'py2puml.domain'))
 ```
 
-* running it (`python3 -m py2puml.example`) outputs the previous PlantUML diagram in the terminal and writes it in a file.
+* running it outputs the previous PlantUML diagram in the terminal and writes it in a file.
 
 
 # Tests
@@ -192,11 +208,12 @@ python3 -m pytest -v
 Code coverage (with [missed branch statements](https://pytest-cov.readthedocs.io/en/latest/config.html?highlight=--cov-branch)):
 
 ```sh
-poetry run pytest -v --cov=py2puml --cov-branch --cov-report term-missing --cov-fail-under 92
+poetry run pytest -v --cov=py2puml --cov-branch --cov-report term-missing --cov-fail-under 93
 ```
 
 # Changelog
 
+* `0.8.0`: added support for union types, and github actions (pre-commit hooks + automated tests)
 * `0.7.2`: added the current working directory to the import path to make py2puml work in any directory or in native virtual environment (not handled by poetry)
 * `0.7.1`: removed obsolete part of documentation: deeply compound types are now well handled (by version `0.7.0`)
 * `0.7.0`: improved the generated PlantUML documentation (added the namespace structure of the code base, homogenized type  between inspection and parsing), improved relationships management (handle forward references, deduplicate relationships)
@@ -224,8 +241,55 @@ Unless stated otherwise all works are licensed under the [MIT license](http://sp
 * [Julien Jerphanion](https://github.com/jjerphan)
 * [Luis Fernando Villanueva Pérez](https://github.com/jonykalavera)
 
+## Pull requests
+
 Pull-requests are welcome and will be processed on a best-effort basis.
-Follow the [contributing guide](CONTRIBUTING.md).
+
+Pull requests must follow the guidelines enforced by the `pre-commit` hooks:
+
+- commit messages must follow the Angular conventions enforced by the `commitlint` hook
+- code formatting must follow the conventions enforced by the `isort` and `yapf` hooks
+- code linting should not detect code smells in your contributions, this is checked by the `ruff` hook
+
+Please also follow the [contributing guide](CONTRIBUTING.md) to ease your contribution.
+
+## Code conventions
+
+The code conventions are described and enforced by [pre-commit hooks](https://pre-commit.com/hooks.html) to maintain consistency across the code base.
+The hooks are declared in the [.pre-commit-config.yaml](.pre-commit-config.yaml) file.
+
+Set the git hooks (pre-commit and commit-msg types):
+
+```sh
+poetry run pre-commit install --hook-type pre-commit --hook-type commit-msg
+```
+
+Before committing, you can check your changes with:
+
+```sh
+# put all your changes in the git staging area
+git add -A
+
+# all hooks
+poetry run pre-commit run --all-files
+
+# a specific hook
+poetry run pre-commit run ruff --all-files
+```
+
+### Commit messages
+
+Please, follow the [conventions of the Angular team](https://github.com/angular/angular/blob/main/CONTRIBUTING.md#-commit-message-format) for commit messages.
+When merging your pull-request, the new version of the project will be derived from the messages.
+
+### Code formatting
+
+This project uses `isort` and `yapf` to format the code.
+The guidelines are expressed in their respective sections in the [pyproject.toml](pyproject.toml) file.
+
+### Best practices
+
+This project uses the `ruff` linter, which is configured in its section in the [pyproject.toml](pyproject.toml) file.
 
 # Current limitations
 

commitlint.config.js

@@ -0,0 +1,8 @@
+module.exports = {
+    extends: [
+        '@commitlint/config-angular'
+    ],
+    rules: {
+        'header-max-length': [2, 'always', 120],
+    }
+}