Skip to content
This repository was archived by the owner on Oct 10, 2026. It is now read-only.

Refactor workspace layout and harden S3/Databricks paths - #7

Merged
anandpant merged 16 commits into
mainfrom
feat/workspace-targets-docs-release-shape
Apr 20, 2026
Merged

anandpant merged 16 commits into
mainfrom
feat/workspace-targets-docs-release-shape

Conversation

@anandpant

@anandpant anandpant commented Apr 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • split the repo into a real Cargo workspace with convex-cdc-core, convex-target-s3, and convex-sync
  • move AWS and Databricks assets under platform/ with explicit S3 and Databricks-native target ownership
  • add Databricks-native bootstrap, Lakeflow AUTO CDC template, and extractor job assets
  • harden the Databricks S3-backed view sync with external-location preflight checks
  • tighten the README and durable docs, remove stale planning docs, add a release-artifacts guide, and drop the repo-local AGENTS.md

Verification

  • cargo fmt --check
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo test --workspace
  • python -m py_compile platform/databricks/native/extractor/convex_cdc_job.py
  • live convex-sync sync-once against the configured Convex deployment
  • live convex-sync run --max-iterations 1 end-to-end through S3 publish
  • live Databricks S3-backed view registration against the real prod prefix
  • live Databricks-native bootstrap SQL apply / checkpoint table smoke test
  • live Databricks serverless notebook smoke run for the native extractor

Notes

  • release automation itself is intentionally left for a follow-up PR; this PR only defines the current release artifact shape in docs/release-artifacts.md
  • the Databricks-native extractor was debugged through several real serverless notebook runs until it completed successfully

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

@anandpant
anandpant merged commit b285548 into main Apr 20, 2026
5 checks passed
@anandpant
anandpant deleted the feat/workspace-targets-docs-release-shape branch April 20, 2026 03:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant