Breaking change: migrate to Pydantic 2.x (#14)

* Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Pydantic 2.3 support * Move to separate components * Move to separate components * Move to separate components * Move to separate components * Move to separate components * Move to separate components * Move to separate components * Remove warnings * Remove warnings * 19 failing unit tests * Remove warnings * Remove warnings * Simplify ListOfOne and ListOfTwo * Change unique_list to a Pydantic2 version * Fix list of one and two after unique_conlist change * Change core and core_async tests with PydanticV2 enum validation changes * Fix more tests * Fix more tests * Fix tests * flake8 * flake8 * mypy * Support Python 3.9 * Disable rapidjson * Latest pydantic * Latest pydantic * Fix * Fix unit test for Pydantic 2.4.2 * Serialize ValueError * Fix * Pydantic 2.4 changes * isort * Review comments * Replace isort/flake8 with ruff, pre-commit autoupdate * Refactor choice/choice_list and enable schema tests * Fix contact person list schema * Adapt expected json schema in test_display_subscription.py and test_migration_summary.py to the actual json schema * Fix pydantic warning: change Accept to return a string as per its schema * Expose types in validators/__init__.py Cleanup validators Move remove_empty_items to core * Bump version from to 1.0.0a0 * Fix ruff errors * Remove UniqueConstrainedList, refactor ListOfOne/ListOfTwo/unique_conlist/choice_list * Fix validate_unique_list for non-hashable items * Add more-itertools to requirements * Breaking change: move ReadOnlyField to validators and refactor it to an Annotated Literal * Pin mypy 1.6.1 * ReadOnlyField support for None and lists * Add ReadOnlyField testcase for list of BaseModels * Add todos * Add UUID testcase for Choice * Bump pydantic to 2.5 * Bump version to 1.0.0a1 * Filter empty items in contact_person_list validator * Cleanup commented code * Convert Pydantic v2's validation errors to our needs * Updated the project version and classifiers * Bump version to 1.0.0a2 * Serialize attribute error * Fix typo * Fix Choice schema to always create an enum * Fix single choice * Rewrite Accept to use default Enum validation * Link issues * Bump version --------- Co-authored-by: Maurits Rijk <[email protected]> Co-authored-by: Tjeerd.Verschragen <[email protected]> Co-authored-by: Peter Boers <[email protected]>
workfloworchestrator · Dec 15, 2023 · fa9630d · fa9630d
1 parent 55245ef
commit fa9630d
Show file tree

Hide file tree

Showing 55 changed files with 2,181 additions and 1,488 deletions.
diff --git a/.bumpversion.cfg b/.bumpversion.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.2.0
+current_version = 1.0.0rc1
 commit = False
 tag = False
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)((\-rc)(?P<build>\d+))?

diff --git a/.github/workflows/run-linting-tests.yml b/.github/workflows/run-linting-tests.yml
@@ -31,12 +31,10 @@ jobs:
           flit install --deps develop --extras fastapi
       - name: Check formatting
         run: |
-          isort -c .
           python -V | grep 10 || black --check .
-      - name: Lint with flake8
+      - name: Lint with ruff
         run: |
-          # stop the build if there are Python syntax errors or undefined names
-          flake8 .
+          ruff .
       - name: Check with mypy
         run: |
           mypy pydantic_forms
diff --git a/.github/workflows/run-unit-tests.yml b/.github/workflows/run-unit-tests.yml
@@ -10,11 +10,8 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.9', '3.10', '3.11']
-        pydantic-version: ['1.9.0', '1.10.0', 'lockfile']
-        fastapi-version: ['0.80.0', '0.90.0', 'lockfile']
-        exclude:
-          - python-version: '3.11'
-            pydantic-version: '1.9.0'
+        pydantic-version: ['2.5.1', 'lockfile']
+        fastapi-version: ['0.103.2', 'lockfile']
       fail-fast: false
     container: python:${{ matrix.python-version }}-slim
     steps:

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,20 +1,22 @@
 repos:
-  - repo: https://github.com/timothycrosley/isort
-    rev: 5.12.0
-    hooks:
-      - id: isort
   - repo: https://github.com/psf/black
-    rev: 23.3.0
+    rev: 23.10.0
     hooks:
       - id: black
         language_version: python3.9
   - repo: https://github.com/asottile/blacken-docs
-    rev: 1.13.0
+    rev: 1.16.0
     hooks:
       - id: blacken-docs
         additional_dependencies: [black==23.1.0]
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    # Ruff version.
+    rev: v0.1.1
+    hooks:
+      - id: ruff
+        args: [ --fix, --exit-non-zero-on-fix, --show-fixes ]
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.4.0
+    rev: v4.5.0
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
@@ -25,23 +27,8 @@ repos:
       - id: debug-statements
       - id: requirements-txt-fixer
       - id: detect-private-key
-  - repo: https://github.com/PyCQA/flake8
-    rev: 6.0.0
-    hooks:
-      - id: flake8
-        additional_dependencies:
-          - flake8-bandit
-          - flake8-bugbear
-          - flake8-comprehensions
-          - flake8-docstrings
-          - flake8-logging-format
-          - flake8-pep3101
-          - flake8-print
-          - flake8-rst
-          - flake8-rst-docstrings
-          - flake8-tidy-imports
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.2.0
+    rev: v1.6.1
     hooks:
       - id: mypy
         language_version: python3.9
@@ -63,6 +50,6 @@ repos:
       - id: python-check-mock-methods
       - id: rst-backticks
   - repo: https://github.com/shellcheck-py/shellcheck-py
-    rev: v0.9.0.2
+    rev: v0.9.0.6
     hooks:
       - id: shellcheck
diff --git a/README.md b/README.md
@@ -3,27 +3,17 @@
 [![pypi_version](https://img.shields.io/pypi/v/pydantic-forms?color=%2334D058&label=pypi%20package)](https://pypi.org/project/pydantic-forms)
 [![Supported python versions](https://img.shields.io/pypi/pyversions/pydantic-forms.svg?color=%2334D058)](https://pypi.org/project/pydantic-forms)
 
-A python package that lets you add smart forms to [FastAPI](https://fastapi.tiangolo.com/)
+A Python package that lets you add smart forms to [FastAPI](https://fastapi.tiangolo.com/)
 and [Flask](https://palletsprojects.com/p/flask/). Forms will respond with a JSON scheme that
-contains all info needed in a Recat frontend with uniforms to render the forms and handle all validation tasks.
+contains all info needed in a React frontend with uniforms to render the forms and handle all validation tasks.
 
-Forms can also consist out of wizard; so you can create complex form flows consisting out of multiple
+Forms can also consist out of a wizard, so you can create complex form flows consisting out of multiple
 consecutive forms. The forms and the validation logic are defined by
 using [Pydantic](https://pydantic-docs.helpmanual.io/) models.
 
 Documentation regarding the usage of Forms can be found
 [here](https://github.com/workfloworchestrator/orchestrator-core/blob/main/docs/architecture/application/forms.md)
 
-### Todo
-
-This package is not ready for use in other projects. To make it production ready:
-
-- [ ] Add Flask example; a reference implementation is available in
-  the [pricelist-backend](https://github.com/acidjunk/pricelist-backend/blob/master/server/main.py)
-- [ ] Setup docs boilerplate
-- [ ] Copy orchestrator core form docs
-- [x] Publish package
-
 ### Installation (Development standalone)
 Install the project and its dependencies to develop on the code.
 
@@ -60,13 +50,13 @@ Run the unit-test suite to verify a correct setup.
 
 #### Step 2 - Run tests
 ```shell
-pytest test/unit_tests
+pytest tests/unit_tests
 ```
 
 or with xdist:
 
 ```shell
-pytest -n auto test/unit_tests
+pytest -n auto tests/unit_tests
 ```
 
 If you do not encounter any failures in the test, you should be able to develop features in the pydantic-forms.

diff --git a/pydantic_forms/__init__.py b/pydantic_forms/__init__.py
@@ -13,4 +13,4 @@
 
 """This is the pydantic-forms engine."""
 
-__version__ = "0.2.0"
+__version__ = "1.0.0rc1"
diff --git a/pydantic_forms/core/__init__.py b/pydantic_forms/core/__init__.py
@@ -10,14 +10,13 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-from pydantic_forms.core.shared import DisplayOnlyFieldType, FormPage, ReadOnlyField, list_forms, register_form
+from pydantic_forms.core.shared import DisplayOnlyFieldType, FormPage, list_forms, register_form
 from pydantic_forms.core.sync import generate_form, post_form, start_form
 
 __all__ = [
     "list_forms",
     "register_form",
     "FormPage",
-    "ReadOnlyField",
     "DisplayOnlyFieldType",
     "post_form",
     "start_form",

diff --git a/pydantic_forms/core/asynchronous.py b/pydantic_forms/core/asynchronous.py
@@ -12,7 +12,7 @@
 # limitations under the License.
 from copy import deepcopy
 from inspect import isasyncgenfunction
-from typing import Any, List, Union
+from typing import Any, Union
 
 import structlog
 from pydantic import ValidationError
@@ -25,7 +25,7 @@
 
 
 async def generate_form(
-    form_generator: Union[StateInputFormGeneratorAsync, None], state: State, user_inputs: List[State]
+    form_generator: Union[StateInputFormGeneratorAsync, None], state: State, user_inputs: list[State]
 ) -> Union[State, None]:
     """Generate form using form generator as defined by a form definition."""
     try:
@@ -40,7 +40,7 @@ async def generate_form(
 
 
 async def post_form(
-    form_generator: Union[StateInputFormGeneratorAsync, None], state: State, user_inputs: List[State]
+    form_generator: Union[StateInputFormGeneratorAsync, None], state: State, user_inputs: list[State]
 ) -> State:
     """Post user_input based ond serve a new form if the form wizard logic dictates it."""
     # there is no form_generator so we return no validated data
@@ -67,10 +67,10 @@ async def post_form(
         try:
             form_validated_data = generated_form(**user_input)
         except ValidationError as e:
-            raise FormValidationError(e.model.__name__, e.errors()) from e  # type: ignore
+            raise FormValidationError(generated_form.__name__, e) from e  # type: ignore
 
         # Update state with validated_data
-        current_state.update(form_validated_data.dict())
+        current_state.update(form_validated_data.model_dump())
 
         # Make next form
         generated_form = await generator.asend(form_validated_data)
@@ -85,7 +85,7 @@ async def post_form(
         return generated_form
 
     # Form is not completely filled raise next form
-    raise FormNotCompleteError(generated_form.schema())
+    raise FormNotCompleteError(generated_form.model_json_schema())
 
 
 def _get_form(key: str) -> StateInputFormGeneratorAsync:
@@ -100,7 +100,7 @@ def _get_form(key: str) -> StateInputFormGeneratorAsync:
 
 async def start_form(
     form_key: str,
-    user_inputs: Union[List[State], None] = None,
+    user_inputs: Union[list[State], None] = None,
     user: str = "Just a user",  # Todo: check if we need users inside form logic?
     **extra_state: Any,
 ) -> State:

diff --git a/pydantic_forms/core/shared.py b/pydantic_forms/core/shared.py
@@ -11,59 +11,49 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 from inspect import isasyncgenfunction, isgeneratorfunction
-from typing import Any, Callable, Dict, Generator, List, Optional
+from typing import Any, Callable
 
 import structlog
-from pydantic import BaseModel, Extra, Field
-from pydantic.fields import ModelField, Undefined
-
-from pydantic_forms.utils.json import json_dumps, json_loads
+from pydantic import BaseModel, ConfigDict
 
 logger = structlog.get_logger(__name__)
 
 
 class DisplayOnlyFieldType:
-    @classmethod
-    def __get_validators__(cls) -> Generator:
-        yield cls.nothing
-
-    def nothing(cls, v: Any, field: ModelField) -> Any:
-        return field.default
+    pass
 
 
 class FormPage(BaseModel):
-    class Config:
-        json_loads = json_loads
-        json_dumps = json_dumps
-        title = "unknown"
-        extra = Extra.forbid
-        validate_all = True
+    model_config = ConfigDict(
+        arbitrary_types_allowed=True,
+        title="unknown",
+        extra="forbid",
+        validate_default=True,
+    )
+
+    def __init__(self, **data: Any):
+        frozen_fields = {k: v for k, v in self.model_fields.items() if v.frozen}
 
-    def __init_subclass__(cls, /, **kwargs: Any) -> None:
-        super().__init_subclass__(**kwargs)
+        def get_value(k: str, v: Any) -> Any:
+            if k in frozen_fields:
+                return frozen_fields[k].default
+            return v
 
+        mutable_data = {k: get_value(k, v) for k, v in data.items()}
+        super().__init__(**mutable_data)
+
+    @classmethod
+    def __pydantic_init_subclass__(cls, /, **kwargs: Any) -> None:
         # The default and requiredness of a field is not a property of a field
         # In the case of DisplayOnlyFieldTypes, we do kind of want that.
         # Using this method we set the right properties after the form is created
-        for field in cls.__fields__.values():
-            try:
-                if issubclass(field.type_, DisplayOnlyFieldType):
-                    field.required = False
-                    field.allow_none = True
-            except TypeError:
-                pass
-
 
-def ReadOnlyField(
-    default: Any = Undefined,
-    *,
-    const: Optional[bool] = None,
-    **extra: Any,
-) -> Any:
-    return Field(default, const=True, uniforms={"disabled": True, "value": default}, **extra)  # type: ignore
+        for field in cls.model_fields.values():
+            if field.frozen:
+                field.validate_default = False
 
 
-FORMS: Dict[str, Callable] = {}
+FORMS: dict[str, Callable] = {}
 
 
 def register_form(key: str, form: Callable) -> None:
@@ -77,5 +67,5 @@ def register_form(key: str, form: Callable) -> None:
     FORMS[key] = form
 
 
-def list_forms() -> List[str]:
+def list_forms() -> list[str]:
     return list(FORMS.keys())
diff --git a/pydantic_forms/core/sync.py b/pydantic_forms/core/sync.py
@@ -12,7 +12,7 @@
 # limitations under the License.
 from copy import deepcopy
 from inspect import isgeneratorfunction
-from typing import Any, Dict, List, Union
+from typing import Any, Union
 
 import structlog
 from pydantic import ValidationError
@@ -25,7 +25,7 @@
 
 
 def generate_form(
-    form_generator: Union[StateInputFormGenerator, None], state: State, user_inputs: List[State]
+    form_generator: Union[StateInputFormGenerator, None], state: State, user_inputs: list[State]
 ) -> Union[State, None]:
     """Generate form using form generator as defined by a form definition."""
     try:
@@ -39,7 +39,7 @@ def generate_form(
     return None
 
 
-def post_form(form_generator: Union[StateInputFormGenerator, None], state: State, user_inputs: List[State]) -> State:
+def post_form(form_generator: Union[StateInputFormGenerator, None], state: State, user_inputs: list[State]) -> State:
     """Post user_input based ond serve a new form if the form wizard logic dictates it."""
     # there is no form_generator so we return no validated data
     if not form_generator:
@@ -65,21 +65,21 @@ def post_form(form_generator: Union[StateInputFormGenerator, None], state: State
             try:
                 form_validated_data = generated_form(**user_input)
             except ValidationError as e:
-                raise FormValidationError(e.model.__name__, e.errors()) from e  # type: ignore
+                raise FormValidationError(generated_form.__name__, e) from e  # type: ignore
 
             # Update state with validated_data
-            current_state.update(form_validated_data.dict())
+            current_state.update(form_validated_data.model_dump())
 
             # Make next form or trigger StopIteration
             generated_form = generator.send(form_validated_data)
 
         # Form is not completely filled; raise next form
-        raise FormNotCompleteError(generated_form.schema())
+        raise FormNotCompleteError(generated_form.model_json_schema())
     except StopIteration as e:
         if user_inputs:
             raise FormOverflowError(f"Did not process all user_inputs ({len(user_inputs)} remaining)")
 
-        # Form is completely filled so we can return the last of the data and
+        # Form is completely filled, so we can return the last of the data and
         return e.value
 
 
@@ -95,9 +95,9 @@ def _get_form(key: str) -> StateInputFormGenerator:
 
 def start_form(
     form_key: str,
-    user_inputs: Union[List[State], None] = None,
+    user_inputs: Union[list[State], None] = None,
     user: str = "Just a user",  # Todo: check if we need users inside form logic?
-    **extra_state: Dict[str, Any],
+    **extra_state: dict[str, Any],
 ) -> State:
     """Handle the logic for the endpoint that the frontend uses to render a form with or without prefilled input.
Original file line number	Diff line number	Diff line change
Expand Up		@@ -13,4 +13,4 @@

		"""This is the pydantic-forms engine."""

		__version__ = "0.2.0"
		__version__ = "1.0.0rc1"