this

bronifty · Oct 20, 2024 · c941308 · c941308
1 parent 5ff1e0a
commit c941308
Show file tree

Hide file tree

Showing 5 changed files with 287 additions and 1 deletion.
diff --git a/docs/opinion/codes/_meta.json b/docs/opinion/codes/_meta.json
@@ -6,5 +6,6 @@
   "react-router",
   "sqlite-wasm",
   "llvm",
-  "collab"
+  "collab",
+  "ryan-blue-tabular-databricks"
 ]
diff --git a/docs/opinion/codes/ryan-blue-tabular-databricks.mdx b/docs/opinion/codes/ryan-blue-tabular-databricks.mdx
@@ -0,0 +1,9 @@
+# Ryan Blue Tabular Databricks
+
+![Ryan Blue Tabular Databricks](./ryan-blue-tabular-databricks/databricks-uniform-delta-iceberg-parquet-snowflake.png)
+
+## Uniform Table Format
+
+Ryan Blue, creator of Iceberg table format, has created a new uniform table format for interoperability between Delta Lake, which is native to Databricks, and Iceberg as well as potentially other formats.
+
+The figure above shows how the metadata catalog (called "unity catalog" in Databricks) can facilitate zero-copy in-place operations between table formats and query engines. In this case, Databricks on Delta is DDL'd into a shared Docker-style volume (represented as a struct mapping the file key to a named location in the catalog) and DML'd as well as DQL'd from the same volume with another, separate query engine (Snowflake).
diff --git a/...-blue-tabular-databricks/databricks-uniform-delta-iceberg-parquet-snowflake.png b/...-blue-tabular-databricks/databricks-uniform-delta-iceberg-parquet-snowflake.png
diff --git a/package.json b/package.json
@@ -16,6 +16,7 @@
     "@types/node": "^16.18.104",
     "aws-cdk-lib": "^2.151.0",
     "ts-node": "^10.9.2",
+    "tsx": "^4.19.1",
     "typescript": "^5.5.4"
   },
   "packageManager": "[email protected]+sha512.dc09430156b427f5ecfc79888899e1c39d2d690f004be70e05230b72cb173d96839587545d09429b55ac3c429c801b4dc3c0e002f653830a420fa2dd4e3cf9cf"

diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml