Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert both list and dict columns to json #1294

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

austinweisgrau
Copy link
Collaborator

In #1143, an update was made that enabled the BigQuery connector to automatically load dict columns as JSON strings rather than fail to load and error out.

List columns need the same fix - they cannot load by default into BigQuery and need to be converted to a JSON string. This PR makes that update.

Copy link

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  parsons/google
  google_bigquery.py
Project Total  

This report was generated by python-coverage-comment-action

@@ -794,7 +794,7 @@ def copy(
quote: Optional[str] = None,
schema: Optional[List[dict]] = None,
max_timeout: int = 21600,
convert_dict_columns_to_json: bool = True,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change because it renames a parameter. I don't know that it will impact many people because the parameter was only added four months ago - it might be that you're the only one using it - and if we are going to play fast and loose with the API I guess now's the time to do it. But alternatively you could just add a second parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants