Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define JSON Schema for construct.yml #943

Open
wants to merge 30 commits into
base: main
Choose a base branch
from

Conversation

jaimergp
Copy link
Contributor

@jaimergp jaimergp commented Feb 28, 2025

Description

This PR converts our venerable construct.py "schema" to a proper JSON Schema. The Schema is built with Pydantic at "commit time" as we do with docs regeneration. At runtime, only the lightweight jsonschema package is used for validation.

With this change we have:

  • IDE autocompletion (examples have been edited to point to local copy; should just work if you open it in VS Code). I'll also submit it to schemastore to associate construct.yml to the schema in the repo
  • Accurate type validation with detailed error reports
  • Self documented schema

However we have lost the ability to report type information in the docs (the autogenerated annotations are too wordy and noisy). I'm not too worried about this because the information was sometimes inaccurate anyway. Instead users are encouraged to use the schema directly, either in their IDE, or exploring it interactively in apps like https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2Fjaimergp%2Fconstructor%2Frefs%2Fheads%2Fpydantic-schema%2Fconstructor%2Fdata%2Fconstructor.schema.json.

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

Additional context:

@bollwyvl mentioned this in a conda-forge Zulip thread and I couldn't resist to attempt a crude conversion. The first commit already gives you inline descriptions and some type validation.

Plenty of work ahead for the very diverse types and actual default values, plus integration with constructor itself.

For fun and laughs, this is the crude script that I used to convert our venerable KEYS "schema" to a Pydantic model:

from constructor.construct import KEYS
from textwrap import indent

print("""
from pydantic import BaseModel, ConfigDict

class ConstructorModel(BaseModel):
    model_config: ConfigDict = ConfigDict(
        extra='forbid',
        use_attribute_docstrings=True,
    )
"""
)

for key, required, type, desc in KEYS:
    type_name = getattr(type, "__name__", getattr(type.__class__, "__name__"))
    print(f"    {key}: {type_name}", "= ..." if required else f"= {type_name}()")
    print('    """')
    print(indent(desc.strip(), "    "))
    print('    """')

@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Feb 28, 2025
@jaimergp jaimergp changed the title Crude schema migration to Pydantic model and JSON Schema (WIP) Define JSON Schema for `construct.yml Mar 5, 2025
@jaimergp jaimergp changed the title Define JSON Schema for `construct.yml Define JSON Schema for construct.yml Mar 5, 2025
@jaimergp jaimergp marked this pull request as ready for review March 6, 2025 09:07
@jaimergp jaimergp requested a review from a team as a code owner March 6, 2025 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed [bot] added once the contributor has signed the CLA
Projects
Status: 🆕 New
Development

Successfully merging this pull request may close these issues.

3 participants