Run AWS test suites only on ready pull requests and related changes

.agents/skills/development-workflow/SKILL.md

@@ -6,7 +6,7 @@
 #
 # SPDX-License-Identifier: MIT
 name: development-workflow
-description: Deliver a PyAthena change through a dedicated worktree, Draft PR, two distinct self-reviews, independent review, and current CI before Ready. Use when implementing or updating a PR, not for a bounded review-only request.
+description: Deliver a PyAthena change through a dedicated worktree, Draft PR, two distinct self-reviews, independent review, and current CI once marked Ready. Use when implementing or updating a PR, not for a bounded review-only request.
 ---
 
 # PyAthena PR delivery
@@ -28,8 +28,11 @@ Keep unrelated worktrees and changes intact.
 4. Complete [self-review](../self-review/SKILL.md), then [self-review-round-two](../self-review-round-two/SKILL.md), fixing verified findings and validating affected behavior.
 5. Collect [independent-review](../independent-review/SKILL.md).
    Repairs pass through both self-review perspectives and an independent follow-up before completion.
-6. Check the current PR with `gh pr view` and `gh pr checks` before `gh pr ready`.
-   Confirm the published head matches the reviewed head, all applicable checks have completed successfully, and the PR has no merge conflict.
+6. Draft PRs run only the offline checks; `gh pr ready` starts the AWS jobs.
+   Before it, confirm the published head matches the reviewed head, the offline checks have completed successfully, and the PR has no merge conflict.
+   After it, check the current PR with `gh pr view` and `gh pr checks`, and confirm all applicable checks have completed successfully.
+   If an AWS job fails, return the PR to Draft with `gh pr ready --undo` until the failure is resolved.
+   To obtain AWS results while the PR is still Draft, dispatch the Test workflow on its branch.
    Pending, cancelled, missing expected checks, and `UNKNOWN` mergeability do not establish readiness.
    Explain intentionally skipped jobs from workflow conditions; a passing rerun of one failed job does not make the remaining failures pass.
    Keep the PR Draft while required review or validation remains incomplete, unless the user explicitly changes that requirement.

.github/workflows/test-suite.yaml

@@ -17,6 +17,14 @@ on:
         description: Maximum number of Python versions tested concurrently
         type: number
         default: 5
+      skip-spark:
+        description: Skip the Spark tests of the PyAthena suite
+        type: boolean
+        default: false
+      skip-sqla:
+        description: Skip the SQLAlchemy tests of the PyAthena suite
+        type: boolean
+        default: false
 
 jobs:
   run:
@@ -26,6 +34,9 @@ jobs:
 
     env:
       TEST_TYPE: ${{ inputs.test-type }}
+      PYTEST_ADDOPTS: >-
+        ${{ inputs.skip-spark && '--ignore=tests/pyathena/spark --ignore=tests/pyathena/aio/spark' || '' }}
+        ${{ inputs.skip-sqla && '--ignore=tests/pyathena/sqlalchemy --ignore=tests/pyathena/aio/sqlalchemy' || '' }}
       AWS_DEFAULT_REGION: us-west-2
       # Keep SDK retries bounded; PyAthena also retries wrapped catalog errors.
       AWS_RETRY_MODE: standard

.github/workflows/test.yaml

@@ -9,14 +9,21 @@ name: Test
 
 on:
   pull_request:
+    # ready_for_review starts the AWS jobs for a pull request leaving Draft;
+    # converted_to_draft starts a run without them, which cancels an
+    # in-progress run through the concurrency group. paths-ignore applies to
+    # both, so a pull request that now changes only docs starts neither.
+    types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]
     paths-ignore:
       - 'docs/**'
       - '**.md'
+  # The scheduled run executes every suite, including the ones that pull
+  # requests only run when related files change.
   schedule:
     - cron:  '0 0 * * 0'
-  # Allows refreshing the README status badge on demand: the badge reflects
-  # the latest run on the default branch, which is otherwise only the weekly
-  # scheduled run and stays red for up to a week after a transient failure.
+  # Runs every suite on the selected branch: before a release, on demand for
+  # a pull request, and to refresh the README status badge after a transient
+  # failure on the default branch.
   workflow_dispatch:
 
 permissions:
@@ -29,20 +36,91 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
-# The three suites create their own schemas and tables, so they run in
-# parallel; each is still a separate job for "Re-run failed jobs".
 jobs:
+  # Offline checks run for every event, including Draft and fork pull requests.
+  lint:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
+        with:
+          python-version: '3.12'
+          enable-cache: true
+      - uses: taiki-e/install-action@7a79fe8c3a13344501c80d99cae481c1c9085912 # v2.81.10
+        with:
+          tool: just
+      - run: just lint
+
+  # Selects the AWS suites. Draft and external-fork pull requests run none.
+  # A ready pull request always runs the PyAthena suite; it runs the
+  # SQLAlchemy tests (the compliance suites and the PyAthena suite's
+  # SQLAlchemy tests) and the Spark tests only when their code, tests,
+  # dependencies, or this workflow change.
+  changes:
+    if: >-
+      github.event_name != 'pull_request' ||
+      (!github.event.pull_request.draft &&
+       github.event.pull_request.head.repo.full_name == github.repository)
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: read
+    outputs:
+      sqla: ${{ steps.filter.outputs.sqla }}
+      spark: ${{ steps.filter.outputs.spark }}
+    steps:
+      - id: filter
+        env:
+          GH_TOKEN: ${{ github.token }}
+          EVENT_NAME: ${{ github.event_name }}
+          REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          if [[ "$EVENT_NAME" != "pull_request" ]]; then
+            echo "sqla=true" >> "$GITHUB_OUTPUT"
+            echo "spark=true" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          files=$(gh api "repos/$REPO/pulls/$PR_NUMBER/files" --paginate --jq '.[].filename')
+          printf 'Changed files:\n%s\n' "$files"
+          shared='^(\.github/workflows/test(-suite)?\.yaml|justfile|pyproject\.toml|uv\.lock)$'
+          sqla="$shared|^pyathena/(aio/)?sqlalchemy/|^tests/sqlalchemy/|^tests/pyathena/(aio/)?sqlalchemy/|^setup\.cfg$"
+          spark="$shared|^pyathena/(aio/)?spark/|^tests/pyathena/(aio/)?spark/"
+          if grep -qE "$sqla" <<< "$files"; then
+            echo "sqla=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "sqla=false" >> "$GITHUB_OUTPUT"
+          fi
+          if grep -qE "$spark" <<< "$files"; then
+            echo "spark=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "spark=false" >> "$GITHUB_OUTPUT"
+          fi
+
+  # The three suites create their own schemas and tables, so they run in
+  # parallel; each is still a separate job for "Re-run failed jobs".
   test:
+    needs: changes
     uses: ./.github/workflows/test-suite.yaml
     with:
       test-type: pyathena
+      skip-spark: ${{ needs.changes.outputs.spark != 'true' }}
+      skip-sqla: ${{ needs.changes.outputs.sqla != 'true' }}
 
   test-sqla:
+    needs: changes
+    if: needs.changes.outputs.sqla == 'true'
     uses: ./.github/workflows/test-suite.yaml
     with:
       test-type: sqla
 
   test-sqla-async:
+    needs: changes
+    if: needs.changes.outputs.sqla == 'true'
     uses: ./.github/workflows/test-suite.yaml
     with:
       test-type: sqla_async

AGENTS.md

@@ -23,7 +23,7 @@ Preserve existing and upstream copyright notices; do not apply the new original-
 - **NEVER** commit directly to `master` — always create a feature branch and PR
 - Create PRs as drafts: `gh pr create --draft`
 - Use a dedicated worktree for each PR and follow the repository's PR template.
-- For PR delivery, follow [development-workflow](.agents/skills/development-workflow/SKILL.md): create the draft, complete two distinct self-reviews, collect an independent review, and check current CI before Ready.
+- For PR delivery, follow [development-workflow](.agents/skills/development-workflow/SKILL.md): create the draft, complete two distinct self-reviews, collect an independent review, then mark the PR Ready to start its AWS CI and confirm the result.
 - Run [self-review](.agents/skills/self-review/SKILL.md) for behavior and implementation, then [self-review-round-two](.agents/skills/self-review-round-two/SKILL.md) for compatibility, operational consequences, and factual claims.
 - Follow [independent-review](.agents/skills/independent-review/SKILL.md) after both rounds; Codex-authored changes use Claude Code `claude-fable-5-1`, profile `max`, effort `high`, never Enterprise.
 - Post new review records inline on relevant diff lines using the GitHub review API's `comments` array with an empty review body; record repairs with the review-comment reply API.

docs/testing.md

@@ -148,6 +148,28 @@ Sanitize logs before sharing them.
 
 ## GitHub Actions
 
+The Test workflow runs for pull requests that change files other than `docs/` and Markdown.
+It runs the offline checks (`just lint`) on each of them, including Drafts and external forks, and runs the AWS suites as follows:
+
+| Trigger | PyAthena suite | SQLAlchemy tests | Spark tests |
+| --- | --- | --- | --- |
+| Draft pull request | No | No | No |
+| Ready pull request from a branch of this repository | Yes | When related files change | When related files change |
+| Weekly schedule and manual dispatch | Yes | Yes | Yes |
+
+The SQLAlchemy tests are the compliance suites and the PyAthena suite's `tests/pyathena/sqlalchemy/` and `tests/pyathena/aio/sqlalchemy/`.
+The Spark tests are the PyAthena suite's `tests/pyathena/spark/` and `tests/pyathena/aio/spark/`.
+When the SQLAlchemy or Spark tests do not run, the PyAthena suite runs without them.
+For the SQLAlchemy tests, the related files are `pyathena/sqlalchemy/`, `pyathena/aio/sqlalchemy/`, `tests/sqlalchemy/`, their PyAthena suite test directories, and `setup.cfg`.
+For the Spark tests, they are `pyathena/spark/`, `pyathena/aio/spark/`, and their PyAthena suite test directories.
+Changes to `pyproject.toml`, `uv.lock`, `justfile`, or the Test workflows run both.
+For a pull request from a branch of this repository that still changes files other than `docs/` and Markdown, marking the Draft ready for review starts the AWS jobs, and converting it back to Draft cancels AWS jobs still running.
+To run every suite on a branch, dispatch the workflow:
+
+```bash
+gh workflow run test.yaml --ref <branch>
+```
+
 Project policy excludes external-fork pull requests from AWS integration CI.
 Maintainers do not approve those jobs as a substitute for contributor testing.
 Checks without AWS access may still run on a fork pull request.

pyproject.toml

@@ -236,6 +236,7 @@ commands =
     sqla_async: just test sqla-async
 passenv =
     TOXENV
+    PYTEST_ADDOPTS
     AWS_*
     GITHUB_*
 """