Add OpenAPI 3.1.0 support (#856)

Author: dbanty

.changeset/openapi_31_support.md

@@ -0,0 +1,22 @@
+---
+default: minor
+---
+
+# OpenAPI 3.1 support
+
+The generator will now attempt to generate code for OpenAPI documents with versions 3.1.x (previously, it would exit immediately on seeing a version other than 3.0.x). The following specific OpenAPI 3.1 features are now supported:
+
+- `null` as a type
+- Arrays of types (e.g., `type: [string, null]`)
+- `const` (defines `Literal` types)
+
+The generator does not currently validate that the OpenAPI document is valid for a specific version of OpenAPI, so it may be possible to generate code for documents that include both removed 3.0 syntax (e.g., `nullable`) and new 3.1 syntax (e.g., `null` as a type).
+
+Thanks to everyone who helped make this possible with discussions and testing, including:
+
+- @frco9
+- @vogre
+- @naddeoa
+- @staticdev
+- @philsturgeon
+- @johnthagen

.changeset/removed_query_parameter_nullablerequired_special_case.md

@@ -0,0 +1,7 @@
+---
+default: major
+---
+
+# Removed query parameter nullable/required special case
+
+In previous versions, setting _either_ `nullable: true` or `required: false` on a query parameter would act like both were set, resulting in a type signature like `Union[None, Unset, YourType]`. This special case has been removed, query parameters will now act like all other types of parameters.

.gitignore

@@ -27,4 +27,5 @@ htmlcov/
 
 # Generated end to end test data
 my-test-api-client/
-custom-e2e/
+custom-e2e/
+3-1-features-client

CONTRIBUTING.md

@@ -12,8 +12,8 @@ A bug is one of:
 2. The generated code is invalid or incorrect
 3. An error message is unclear or incorrect
 4. Something which used to work no longer works, except:
-  1. Intentional breaking changes, which are documented in the [changelog](https://github.com/openapi-generators/openapi-python-client/blob/main/CHANGELOG.md)
-  2. Breaking changes to unstable features, like custom templates
+   1. Intentional breaking changes, which are documented in the [changelog](https://github.com/openapi-generators/openapi-python-client/blob/main/CHANGELOG.md)
+   2. Breaking changes to unstable features, like custom templates
 
 If your issue does not fall under one of the above, it is not a bug; check out "[Requesting a feature](#requesting-a-feature).
 
@@ -27,14 +27,14 @@ A feature is usually:
 
 1. An improvement to the way the generated code works
 2. A feature of the generator itself which makes its use easier (e.g., a new config option)
-3. **Support for part of the OpenAPI spec**; this generate _does not yet_ support every OpenAPI feature, these missing features **are not bugs**.
+3. **Support for part of the OpenAPI spec**; this generator _does not yet_ support every OpenAPI feature, these missing features **are not bugs**.
 
 To request a feature:
 
 1. Search through [discussions](https://github.com/openapi-generators/openapi-python-client/discussions/categories/feature-request) to see if the feature you want has already been requested. If it has:
-  1. Upvote it with the little arrow on the original post. This enables code contributors to prioritize the most-demanded features.
-  2. Optionally leave a comment describing why _you_ want the feature, if no existing thread already covers your use-case
-3. If a relevant discussion does not already exist, create a new one. If you are not requesting support for part of the OpenAPI spec, **you must** describe _why_ you want the feature. What real-world use-case does it improve? For example, "raise exceptions for invalid responses" might have a description of "it's not worth the effort to check every error case by hand for the one-off scripts I'm writing".
+   1. Upvote it with the little arrow on the original post. This enables code contributors to prioritize the most-demanded features.
+   2. Optionally leave a comment describing why _you_ want the feature, if no existing thread already covers your use-case
+2. If a relevant discussion does not already exist, create a new one. If you are not requesting support for part of the OpenAPI spec, **you must** describe _why_ you want the feature. What real-world use-case does it improve? For example, "raise exceptions for invalid responses" might have a description of "it's not worth the effort to check every error case by hand for the one-off scripts I'm writing".
 
 ## Contributing Code
 
@@ -45,36 +45,41 @@ To request a feature:
 3. Use `poetry install` in the project directory to create a virtual environment with the relevant dependencies.
 4. Enter a `poetry shell` to make running commands easier.
 
-### Writing Code
+### Writing tests
 
-1. Write some code and make sure it's covered by unit tests. All unit tests are in the `tests` directory and the file structure should mirror the structure of the source code in the `openapi_python_client` directory.
+All changes must be tested, I recommend writing the test first, then writing the code to make it pass. 100% code coverage is enforced in CI, a check will fail in GitHub if your code does not have 100% coverage. An HTML report will be added to the test artifacts in this case to help you locate missed lines.
 
-#### Run Checks and Tests
+If you think that some of the added code is not testable (or testing it would add little value), mention that in your PR and we can discuss it.
 
-2. When in a Poetry shell (`poetry shell`) run `task check` in order to run most of the same checks CI runs. This will auto-reformat the code, check type annotations, run unit tests, check code coverage, and lint the code.
+1. If you're adding support for a new OpenAPI feature or covering a new edge case, add an [end-to-end test](#end-to-end-tests)
+2. If you're modifying the way an existing feature works, make sure an existing test generates the _old_ code in `end_to_end_tests/golden-record`. You'll use this to check for the new code once your changes are complete.
+3. If you're improving an error or adding a new error, add a [unit test](#unit-tests)
 
-#### Rework end-to-end tests 
+#### End-to-end tests
 
-3. If you're writing a new feature, try to add it to the end-to-end test.
-   1. If adding support for a new OpenAPI feature, add it somewhere in `end_to_end_tests/openapi.json`
-   2. Regenerate the "golden records" with `task regen`. This client is generated from the OpenAPI document used for end-to-end testing.
-   3. Check the changes to `end_to_end_tests/golden-record` to confirm only what you intended to change did change and that the changes look correct.
-4. **If you added a test above OR modified the templates**: Run the end-to-end tests with `task e2e`. This will generate clients against `end_to_end_tests/openapi.json` and compare them with the golden record. The tests will fail if **anything is different**. The end-to-end tests are not included in `task check` as they take longer to run and don't provide very useful feedback in the event of failure. If an e2e test does fail, the easiest way to check what's wrong is to run `task regen` and check the diffs. You can also use `task re` which will run `regen` and `e2e` in that order.
+This project aims to have all "happy paths" (types of code which _can_ be generated) covered by end to end tests (snapshot tests). In order to check code changes against the previous set of snapshots (called a "golden record" here), you can run `poetry run task e2e`. To regenerate the snapshots, run `poetry run task regen`.
 
+There are 4 types of snapshots generated right now, you may have to update only some or all of these depending on the changes you're making. Within the `end_to_end_tets` directory:
 
-### Creating a Pull Request
+1. `baseline_openapi_3.0.json` creates `golden-record` for testing OpenAPI 3.0 features
+2. `baseline_openapi_3.1.yaml` is checked against `golden-record` for testing OpenAPI 3.1 features (and ensuring consistency with 3.0)
+3. `test_custom_templates` are used with `baseline_openapi_3.0.json` to generate `custom-templates-golden-record` for testing custom templates
+4. `3.1_specific.openapi.yaml` is used to generate `test-3-1-golden-record` and test 3.1-specific features (things which do not have a 3.0 equivalent)
+
+#### Unit tests
 
-Once you've written the code and run the checks, the next step is to create a pull request against the `main` branch of this repository. This repository uses [conventional commits] squashed on each PR, then uses [Knope] to auto-generate CHANGELOG.md entries for release. So the title of your PR should be in the format of a conventional commit written in plain english as it will end up in the CHANGELOG. Some example PR titles:
+> **NOTE**: Several older-style unit tests using mocks exist in this project. These should be phased out rather than updated, as the tests are brittle and difficult to maintain. Only error cases should be tests with unit tests going forward.
 
-- feat: Support for `allOf` in OpenAPI documents (closes #123).
-- refactor!: Removed support for Python 3.5
-- fix: Data can now be passed to multipart bodies along with files.
+In some cases, we need to test things which cannot be generated—like validating that errors are caught and handled correctly. These should be tested via unit tests in the `tests` directory, using the `pytest` framework.
+
+### Creating a Pull Request
 
-Once your PR is created, a series of automated checks should run. If any of them fail, try your best to fix them.
+Once you've written the tests and code and run the checks, the next step is to create a pull request against the `main` branch of this repository. This repository uses [Knope] to auto-generate release notes and version numbers. This can either be done by setting the title of the PR to a [conventional commit] (for simple changes) or by adding [changesets]. If the changes are not documented yet, a check will fail on GitHub. The details of this check will have suggestions for documenting the change (including an example change file for changesets).
 
 ### Wait for Review
 
 As soon as possible, your PR will be reviewed. If there are any changes requested there will likely be a bit of back and forth. Once this process is done, your changes will be merged into main and included in the next release. If you need your changes available on PyPI by a certain time, please mention it in the PR, and we'll do our best to accommodate.
 
-[Conventional Commits]: https://www.conventionalcommits.org/en/v1.0.0/
-[Knope]: https://knope-dev.github.io/knope/
+[Knope]: https://knope.tech
+[changesets]: https://knope.tech/reference/concepts/changeset/
+[Conventional Commits]: https://knope.tech/reference/concepts/conventional-commits/

README.md

@@ -7,7 +7,7 @@
 
 # openapi-python-client
 
-Generate modern Python clients from OpenAPI 3.0 documents.
+Generate modern Python clients from OpenAPI 3.0 and 3.1 documents.
 
 _This generator does not support OpenAPI 2.x FKA Swagger. If you need to use an older document, try upgrading it to
 version 3 first with one of many available converters._
@@ -69,25 +69,16 @@ _Be forewarned, this is a beta-level feature in the sense that the API exposed i
 ## What You Get
 
 1. A `pyproject.toml` file with some basic metadata intended to be used with [Poetry].
-1. A `README.md` you'll most definitely need to update with your project's details
-1. A Python module named just like the auto-generated project name (e.g. "my_api_client") which contains:
+2. A `README.md` you'll most definitely need to update with your project's details
+3. A Python module named just like the auto-generated project name (e.g. "my_api_client") which contains:
    1. A `client` module which will have both a `Client` class and an `AuthenticatedClient` class. You'll need these
       for calling the functions in the `api` module.
-   1. An `api` module which will contain one module for each tag in your OpenAPI spec, as well as a `default` module
+   2. An `api` module which will contain one module for each tag in your OpenAPI spec, as well as a `default` module
       for endpoints without a tag. Each of these modules in turn contains one function for calling each endpoint.
-   1. A `models` module which has all the classes defined by the various schemas in your OpenAPI spec
+   3. A `models` module which has all the classes defined by the various schemas in your OpenAPI spec
 
-For a full example you can look at the `end_to_end_tests` directory which has an `openapi.json` file.
-"golden-record" in that same directory is the generated client from that OpenAPI document.
-
-## OpenAPI features supported
-
-1. All HTTP Methods
-1. JSON and form bodies, path and query parameters
-1. File uploads with multipart/form-data bodies
-1. float, string, int, date, datetime, string enums, and custom schemas or lists containing any of those
-1. html/text or application/json responses containing any of the previous types
-1. Bearer token security
+For a full example you can look at the `end_to_end_tests` directory which has `baseline_openapi_3.0.json` and `baseline_openapi_3.1.yaml` files.
+The "golden-record" in that same directory is the generated client from either of those OpenAPI documents.
 
 ## Configuration
 
@@ -98,7 +89,7 @@ The following parameters are supported:
 
 Used to change the name of generated model classes. This param should be a mapping of existing class name
 (usually a key in the "schemas" section of your OpenAPI document) to class_name and module_name. As an example, if the
-name of the a model in OpenAPI (and therefore the generated class name) was something like "\_PrivateInternalLongName"
+name of a model in OpenAPI (and therefore the generated class name) was something like "_PrivateInternalLongName"
 and you want the generated client's model to be called "ShortName" in a module called "short_name" you could do this:
 
 Example:
@@ -110,7 +101,7 @@ class_overrides:
     module_name: short_name
 ```
 
-The easiest way to find what needs to be overridden is probably to generate your client and go look at everything in the models folder.
+The easiest way to find what needs to be overridden is probably to generate your client and go look at everything in the `models` folder.
 
 ### project_name_override and package_name_override
 

end_to_end_tests/3.1_specific.openapi.yaml

@@ -0,0 +1,49 @@
+openapi: "3.1.0"
+info:
+  title: "Test 3.1 Features"
+  description: "Test new OpenAPI 3.1 features"
+  version: "0.1.0"
+paths:
+  "/const/{path}":
+    post:
+      tags: [ "const" ]
+      parameters:
+        - in: "path"
+          required: true
+          schema:
+            const: "this goes in the path"
+          name: "path"
+        - in: "query"
+          required: true
+          schema:
+            const: "this always goes in the query"
+          name: "required query"
+        - in: "query"
+          schema:
+            const: "this sometimes goes in the query"
+          name: "optional query"
+      requestBody:
+        required: true
+        content:
+          "application/json":
+            schema:
+              type: object
+              properties:
+                required:
+                  const: "this always goes in the body"
+                optional:
+                  const: "this sometimes goes in the body"
+                nullable:
+                  oneOf:
+                    - type: "null"
+                    - const: "this or null goes in the body"
+              required:
+                - required
+                - nullable
+      responses:
+        "200":
+          description: "Successful Response"
+          content:
+            "application/json":
+              schema:
+                const: "Why have a fixed response? I dunno"

end_to_end_tests/openapi.json end_to_end_tests/baseline_openapi_3.0.json

@@ -1,5 +1,5 @@
 {
-  "openapi": "3.0.2",
+  "openapi": "3.0.3",
   "info": {
     "title": "My Test API",
     "description": "An API for testing openapi-python-client",
@@ -449,10 +449,10 @@
         }
       }
     },
-    "/tests/defaults": {
+    "/defaults": {
       "post": {
         "tags": [
-          "tests"
+          "defaults"
         ],
         "summary": "Defaults",
         "operationId": "defaults_tests_defaults_post",
@@ -467,6 +467,16 @@
             "name": "string_prop",
             "in": "query"
           },
+          {
+            "required": true,
+            "schema": {
+              "title": "String with num default",
+              "type": "string",
+              "default": 1
+            },
+            "name": "string with num",
+            "in": "query"
+          },
           {
             "required": true,
             "schema": {
@@ -2425,7 +2435,8 @@
         "in": "header",
         "required": false,
         "schema": {
-          "type": "string"
+          "type": "string",
+          "nullable": true
         }
       },
       "cookie-param": {