Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add JSON type, bindparam support #1147

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

r1b
Copy link

@r1b r1b commented Dec 10, 2024

Extracted from #1146, this is a smaller set of changes that establish a JSON type.

Changes:

  • Export a JSON type
  • Accept json_serializer / json_deserializer dialect kwargs
  • Handle JSON bind parameters
  • Tests

Once this is merged, users will be able to:

  • Define tables with JSON columns
  • Reference JSON columns in expressions
  • Submit JSON query parameters (as STRINGs, with output conversion back to JSON)

In future PRs, we can add support for:

  • JSON comparator conversion functions (BOOL / LAX_BOOL, etc.)
  • JSON indexing operations

Fixes #399 🦕

@r1b r1b requested review from a team as code owners December 10, 2024 23:38
@r1b r1b requested a review from alvarowolfx December 10, 2024 23:38
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. labels Dec 10, 2024
Comment on lines +5 to +8
def bind_expression(self, bindvalue):
# JSON query parameters have type STRING
# This hook ensures that the rendered expression has type JSON
return sqlalchemy.func.PARSE_JSON(bindvalue, type_=self)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note An alternative here would be to just preserve the STRING type and let BigQuery handle the cast, but this seems hard to reason about. It feels right to have the expression type in BigQuery match the expression type here in SQLAlchemy.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at [1] and pondering this - it wouldn't be that surprising if a user wanted to submit invalid JSON as a query parameter at some point. This could be supported by parameterizing this behavior in the type, i.e: JSON(strict=False) or similar. Not worried about it right now, though.

[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions#non-validation_of_string_inputs

Comment on lines +1100 to +1101
json_serializer=None,
json_deserializer=None,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note See https://docs.sqlalchemy.org/en/20/core/type_basics.html#sqlalchemy.types.JSON section "Customizing the JSON Serializer"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for JSON datatype
2 participants