.travis.yml: run full tests only on changed subtrees

Previously, we would run full (as opposed to short) tests on
every commit. This CL tries to be more selective: we still run
short tests on every commit, but only run full tests on subtrees
that have changed.

This will not only speed up tests, but more importantly, it will
reduce the number of builds that fail due to flaky tests, since
a flaky test (which should never be a short test) will run only
when its package changes.

We still run full tests on the whole repo when certain files or
directories change, like the .travis.yml file itself.

See googleapis#633.

Change-Id: Ic87b9737409f96fd396302bc7f42f5182f94e1f3
Reviewed-on: https://code-review.googlesource.com/13333
Reviewed-by: Chris Broadfoot <[email protected]>

.travis.yml

@@ -9,7 +9,7 @@ install:
 script:
 - openssl aes-256-cbc -K $encrypted_a8b3f4fc85f4_key -iv $encrypted_a8b3f4fc85f4_iv -in key.json.enc -out key.json -d
 - GCLOUD_TESTS_GOLANG_PROJECT_ID="dulcet-port-762" GCLOUD_TESTS_GOLANG_KEY="$(pwd)/key.json"
-  go test -race -v cloud.google.com/go/...
+  ./run-tests.sh $TRAVIS_COMMIT
 env:
   matrix:
     # The GCLOUD_TESTS_API_KEY environment variable.

run-tests.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# Selectively run tests for this repo, based on what has changed
+# in a commit. Runs short tests for the whole repo, and full tests
+# for changed directories.
+
+set -e
+
+prefix=cloud.google.com/go
+
+dryrun=false
+if [[ $1 == "-n" ]]; then
+  dryrun=true
+  shift
+fi
+
+if [[ $1 == "" ]]; then
+  echo >&2 "usage: $0 [-n] COMMIT"
+  exit 1
+fi
+
+# Files or directories that cause all tests to run if modified.
+declare -A run_all
+run_all=([.travis.yml]=1 [run-tests.sh]=1)
+
+function run {
+  if $dryrun; then
+    echo $*
+  else
+    (set -x; $*)
+  fi
+}
+
+
+# Find all the packages that have changed in this commit.
+declare -A changed_packages
+
+for f in $(git diff-tree --no-commit-id --name-only -r $1); do
+  if [[ ${run_all[$f]} == 1 ]]; then
+    # This change requires a full test. Do it and exit.
+    run go test -race -v $prefix/...
+    exit
+  fi
+  # Map, e.g., "spanner/client.go" to "$prefix/spanner".
+  d=$(dirname $f)
+  if [[ $d == "." ]]; then
+    pkg=$prefix
+  else
+    pkg=$prefix/$d
+  fi
+  changed_packages[$pkg]=1
+done
+
+echo "changed packages: ${!changed_packages[*]}"
+
+
+# Reports whether its argument, a package name, depends (recursively)
+# on a changed package.
+function depends_on_changed_package {
+  # According to go list, a package does not depend on itself, so
+  # we test that separately.
+  if [[ ${changed_packages[$1]} == 1 ]]; then
+    return 0
+  fi
+  for dep in $(go list -f '{{range .Deps}}{{.}} {{end}}' $1); do
+    if [[ ${changed_packages[$dep]} == 1 ]]; then
+      return 0
+    fi
+  done
+  return 1
+}
+
+# Collect the packages into two separate lists. (It is faster go test a list of
+# packages than to individually go test each one.)
+
+shorts=
+fulls=
+for pkg in $(go list ./...); do            # for each package in the repo
+  if depends_on_changed_package $pkg; then # if it depends on a changed package
+    fulls="$fulls $pkg"                    # run the full test
+  else                                     # otherwise
+    shorts="$shorts $pkg"                  # run the short test
+  fi
+done
+run go test -race -v -short $shorts
+run go test -race -v $fulls