Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add semi_structured module #75

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Rajat-Sirohi
Copy link

This PR is a prototype of a larger project idea to have Gen3 support a data lake architecture. Specifically, I've added support for CRUD endpoints for semi-structured data. Additionally, I've modified the Metadata model to include optional baseid and created_date columns in order to support versioning of semi-structured data records. Version information is accessed via two additional /versions and /latest endpoints. More specific changes are detailed below:

  • Add new semi_structured module
  • Add baseid and created_date columns to Metadata model + corresponding migration script
  • Add check to aliases module which ensures that new alias does not have the same name as an existing GUID
  • Modify update_metadata() to create a record if it doesn't already exist
  • Add unit tests for everything
  • Add pytest-dependency package to poetry.lock (convenient for unit testing)
  • Update documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants