Releases: open-infrastructure-labs/caerus-dike
v1.5.0
v1.4.0
v1.2.0
This release adds support for Spark cluster operation.
- many of the scripts were updated to support operation on a cluster.
- This includes run_tpch.sh, which is now dependent on starting the sparklauncher docker.
- also updated names of spark dockers to include revision number.
- New scripts added for launching master, worker, and launcher using docker host networking.
- start-master-host.sh, start-worker-host.sh, start-launcher-host.sh
- a new config file called: spark.config is included as an example for the host IPs to be used for the above host networking scripts.
- demo.sh does not use a cluster by default.
v2.0.0
This release introduces NDP server using a Rule-based Spark datasource which sends a query description as a connected graph to our NDP engine.
- The Rule-based Spark datasource release uses Spark version 3.2.0
- NDP server has a new API that allows for receiving a connected graph as a query description.
- Project, Filter, and Aggregate pushdown is supported.
- parquet files are supported.
v1.3.0
This release updates our v1 to support the Spark 3.2.1 release.
Also updated this release to make it easier to choose a newer Spark version.
The procedure to update to a different Spark version is now simplified. To update the Spark version, simply change the spark_version file in spark/docker.
v1.1.0
This release updates our v1 to support a newer Spark release 3.2.0.
This version also simplifies the build process so that we only download Spark now and do not build it.
This should substantially decrease the overall time to build this repo.
In the past we needed the Spark build since we relied on code for aggregate pushdown that had not yet made it into a release.
However, this code has now been released, and thus we can eliminate the Spark build.