Skip to content

Commit

Permalink
Update github actions for Delta Kernel
Browse files Browse the repository at this point in the history
Based off of #1785.

Updates our current `test.yaml` to not run when there are only changes in `kernel/`.

Adds a separate action for future `kernel/` tests. Currently this just compiles the code.

### Testing

I merged these changes to my own fork and created these PRs to check

allisonport-db#11 Changes to `core/src/main/scala/org/apache/spark/sql/delta/Snapshot.scala`
- https://github.com/allisonport-db/delta/actions/runs/5125555163/jobs/9218878662?pr=11
- https://github.com/allisonport-db/delta/actions/runs/5125555163/jobs/9218878794?pr=11

allisonport-db#12 Changes to `build.sbt`
- https://github.com/allisonport-db/delta/actions/runs/5125557362/jobs/9218883935?pr=12
- https://github.com/allisonport-db/delta/actions/runs/5125557362/jobs/9218884043?pr=12

allisonport-db#13 Changes to `examples/scala/src/main/scala/example/ChangeDataFeed.scala`
- https://github.com/allisonport-db/delta/actions/runs/5125558258/jobs/9218886849?pr=13
- https://github.com/allisonport-db/delta/actions/runs/5125558258/jobs/9218886959?pr=13

allisonport-db#14 Changes to `run-tests.py`
- https://github.com/allisonport-db/delta/actions/runs/5125560869/jobs/9218892247?pr=14
- https://github.com/allisonport-db/delta/actions/runs/5125560869/jobs/9218892419?pr=14

allisonport-db#15 Changes to `kernel/build.sbt` (Spark tests SHOULD NOT run)
- https://github.com/allisonport-db/delta/actions/runs/5125973036/jobs/9219853307?pr=15 (skipped)
- https://github.com/allisonport-db/delta/actions/runs/5125973036/jobs/9219853403?pr=15 (skipped)

Closes #1786

Signed-off-by: Allison Portis <[email protected]>
GitOrigin-RevId: cd5449a905c057f61f66afd7dfd5e5cb19348270
  • Loading branch information
allisonport-db committed Jun 2, 2023
1 parent a4ceca2 commit 767970f
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 1 deletion.
16 changes: 16 additions & 0 deletions .github/workflows/kernel_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: "Delta Kernel Tests"
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-20.04
env:
SCALA_VERSION: 2.12.15
steps:
- uses: actions/checkout@v3
- name: install java
uses: actions/setup-java@v3
with:
distribution: "zulu"
java-version: "8"
- name: Compile code # TODO: run tests later on
run: cd kernel && build/sbt compile
10 changes: 9 additions & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: "Delta Lake Tests"
name: "Delta Spark Tests"
on: [push, pull_request]
jobs:
test:
Expand All @@ -10,6 +10,12 @@ jobs:
SCALA_VERSION: ${{ matrix.scala }}
steps:
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v4
id: git-diff
with:
PATTERNS: |
**
!kernel/**
- name: install java
uses: actions/setup-java@v3
with:
Expand Down Expand Up @@ -49,7 +55,9 @@ jobs:
pipenv run pip install twine==4.0.1
pipenv run pip install wheel==0.33.4
pipenv run pip install setuptools==41.0.1
if: steps.git-diff.outputs.diff
- name: Run Scala/Java and Python tests
run: |
pipenv run python run-tests.py --coverage
cd examples/scala && build/sbt "++ $SCALA_VERSION compile"
if: steps.git-diff.outputs.diff

0 comments on commit 767970f

Please sign in to comment.