Script and doc changes based on Windows testing (#181)

Author: paulthomson

build/travis/1-install-deps-travis.sh

@@ -26,7 +26,7 @@ GITHUB_RELEASE_TOOL_VERSION="v1.1.0"
 
 if [ "$(uname)" == "Darwin" ];
 then
-  brew install python3 unzip wget
+  brew install python3 unzip
   GITHUB_RELEASE_TOOL_ARCH="darwin_amd64"
   ANDROID_HOST_PLATFORM="darwin"
 fi
@@ -35,19 +35,20 @@ if [ "$(uname)" == "Linux" ];
 then
   sudo add-apt-repository ppa:deadsnakes/ppa -y
   sudo apt-get update -q
-  sudo apt-get install python3.6 unzip wget -y
+  sudo apt-get install python3.6 unzip -y
   GITHUB_RELEASE_TOOL_ARCH="linux_amd64"
   ANDROID_HOST_PLATFORM="linux"
 fi
 
 
 pushd /data/bin
-wget "https://github.com/${GITHUB_RELEASE_TOOL_USER}/github-release/releases/download/${GITHUB_RELEASE_TOOL_VERSION}/github-release_${GITHUB_RELEASE_TOOL_VERSION}_${GITHUB_RELEASE_TOOL_ARCH}.tar.gz"
+curl -Lo "github-release_${GITHUB_RELEASE_TOOL_VERSION}_${GITHUB_RELEASE_TOOL_ARCH}.tar.gz" \
+  "https://github.com/${GITHUB_RELEASE_TOOL_USER}/github-release/releases/download/${GITHUB_RELEASE_TOOL_VERSION}/github-release_${GITHUB_RELEASE_TOOL_VERSION}_${GITHUB_RELEASE_TOOL_ARCH}.tar.gz"
 tar xf "github-release_${GITHUB_RELEASE_TOOL_VERSION}_${GITHUB_RELEASE_TOOL_ARCH}.tar.gz"
 popd
 
 
-# Android SDK (ANDROID_HOST_PLATFORM must be set to linux, darwin, or windows).
+# Android SDK (ANDROID_HOST_PLATFORM must be set to linux, darwin, or windows, and ANDROID_HOME must be set to some directory).
 
 ANDROID_TOOLS_FILENAME="sdk-tools-${ANDROID_HOST_PLATFORM}-4333796.zip"
 ANDROID_NDK_FILENAME="android-ndk-r18b-${ANDROID_HOST_PLATFORM}-x86_64.zip"
@@ -57,23 +58,23 @@ mkdir -p "${ANDROID_HOME}"
 pushd "${ANDROID_HOME}"
 
 # Android: "sdk-tools.zip" "tools":
-wget -q "http://dl.google.com/android/repository/${ANDROID_TOOLS_FILENAME}"
+curl -qo "${ANDROID_TOOLS_FILENAME}" "http://dl.google.com/android/repository/${ANDROID_TOOLS_FILENAME}"
 unzip -q "${ANDROID_TOOLS_FILENAME}"
 rm "${ANDROID_TOOLS_FILENAME}"
 
 # Android "android-ndk.zip" "ndk-bundle"
-wget -q "https://dl.google.com/android/repository/${ANDROID_NDK_FILENAME}"
+curl -qo "${ANDROID_NDK_FILENAME}" "https://dl.google.com/android/repository/${ANDROID_NDK_FILENAME}"
 unzip -q "${ANDROID_NDK_FILENAME}"
 rm "${ANDROID_NDK_FILENAME}"
 mv android-ndk-*/ ndk-bundle
 
 # Android "platform-tools.zip" "platform-tools"
-wget -q "https://dl.google.com/android/repository/${ANDROID_PLATFORM_TOOLS_FILENAME}"
+curl -qo "${ANDROID_PLATFORM_TOOLS_FILENAME}" "https://dl.google.com/android/repository/${ANDROID_PLATFORM_TOOLS_FILENAME}"
 unzip -q "${ANDROID_PLATFORM_TOOLS_FILENAME}"
 rm "${ANDROID_PLATFORM_TOOLS_FILENAME}"
 
 # Android "platforms" and "build-tools"
-echo y | sdkmanager \
+echo y | tools/bin/sdkmanager \
   "platforms;android-26" \
   "build-tools;28.0.2"
 

build/travis/build.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Copyright 2018 The GraphicsFuzz Project Authors
 #

build/travis/check_headers.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Copyright 2018 The GraphicsFuzz Project Authors
 #
@@ -33,6 +33,8 @@ def exclude_dirname(f: str):
         "__pycache__",
         ".externalNativeBuild",
         ".mvn",
+        "cmake-build-debug",
+        "cmake-build-release",
 
     ]
 

build/travis/licenses.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Copyright 2018 The GraphicsFuzz Project Authors
 #

build/travis/release.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Copyright 2018 The GraphicsFuzz Project Authors
 #

docs/android-notes.md

@@ -20,9 +20,11 @@ by following our
 [continuous integration script](../build/travis/1-install-deps-travis.sh);
 the download of the Android SDK starts about halfway down.
 You may need to set some environment variables as specified in the comments.
+On Windows, you can use the Git Bash shell.
 
-Ensure that the `ANDROID_HOME=/path/to/android-sdk` and `ANDROID_NDK_HOME=$ANDROID_HOME/ndk-bundle` environment variables are set;
-you may need to open IntelliJ IDEA from the terminal depending on how
+Ensure that the `ANDROID_HOME=/path/to/android-sdk` and `ANDROID_NDK_HOME=$ANDROID_HOME/ndk-bundle` environment variables are set.
+If using IntelliJ and/or Android Studio,
+you may need to open your IDE from the terminal depending on how
 you set the environment variables.
 
 You should also add the `android-sdk/platform-tools` directory

docs/glsl-fuzz-develop.md

@@ -15,7 +15,7 @@ page](glsl-fuzz-releases.md).
 * [Maven](https://maven.apache.org/)
 * [Python 3.5+](https://www.python.org/)
 
-* For our Android workers: [Android SDK & NDK](https://developer.android.com/studio/#command-tools)
+* For our Android workers: [Android SDK & NDK](android-notes.md)
 * For the Vulkan desktop worker: [Vulkan SDK](https://vulkan.lunarg.com/sdk/home)
 
 > Our *workers* are applications that run on the device you wish to test;
@@ -89,6 +89,9 @@ or read the [walkthrough for a brief overview of using all tools
 
 #### Android
 
+Ensure that the `ANDROID_HOME=/path/to/android-sdk` and
+`ANDROID_NDK_HOME=$ANDROID_HOME/ndk-bundle` environment variables are set.
+
 ```shell
 cd vulkan-worker
 ./gradlew assembleDebug

docs/glsl-fuzz-generate.md

@@ -135,7 +135,7 @@ glsl-generate --seed 0 samples/donors samples/100 10 "100" family_100 work/shade
 # Generate some "Vulkan-compatible" GLSL version 300 es shaders that can be translated to SPIR-V for Vulkan testing.
 glsl-generate --seed 0 --generate_uniform_bindings --max_uniforms 10 samples/donors samples/310es 10 "310 es" family_vulkan work/shaderfamilies
 
-# The lines above will take approx. 1-2 minutes each, and will generate a shader family for every
+# Each line above will take approx. 1 minute, and will generate a shader family for every
 # shader in samples/300es or samples/100:
 ls work/shaderfamilies
 

docs/glsl-fuzz-walkthrough.md

@@ -4,6 +4,9 @@ glsl-fuzz is a testing framework for automatically finding and simplifying bugs
 by generating, running, and reducing GLSL shaders.
 
 In this walkthrough, we will briefly demonstrate most features of glsl-fuzz from start to finish, including our browser-based UI.
+We present commands assuming a Linux/Mac environment,
+but Windows users can adapt the commands or
+use the Git Bash shell.
 
 The use of `glsl-fuzz` can be roughly split into the following steps:
 
@@ -68,6 +71,11 @@ You will need to install Python 3.5 or higher, either:
 * By downloading from [https://www.python.org/downloads/](https://www.python.org/downloads/).
 * By downloading and installing some other Python 3 distribution.
 
+For Windows: most recent installers of Python
+add `py`, a Python launcher, to your path.
+Our scripts attempt to use `py -3 SCRIPT.py` to
+execute `SCRIPT.py` using Python 3.5+.
+
 ### `adb`
 
 The Vulkan worker for Android requires that `adb` is on your path.
@@ -164,7 +172,7 @@ glsl-generate --seed 0 samples/100 samples/donors 10 "100" family_100 work/shade
 # Generate some "Vulkan-compatible" GLSL version 300 es shaders that can be translated to SPIR-V for Vulkan testing.
 glsl-generate --seed 0 --generate-uniform-bindings --max-uniforms 10 samples/310es samples/donors 10 "310 es" family_vulkan work/shaderfamilies
 
-# The lines above will take approx. 1-2 minutes each, and will generate a shader family for every
+# Each line above will take approx. 1 minute, and will generate a shader family for every
 # shader in samples/300es or samples/100:
 ls work/shaderfamilies
 
@@ -388,8 +396,10 @@ the `vulkan-worker-android` app running on the Android device.
 > re-run SPIR-V shaders on the device.
 > [We describe this in more detail below](#running-shaders-from-the-command-line).
 
-The intermediate files are saved to the current directory.
-For example:
+The intermediate files are saved in the current directory under
+a `WORKERNAME/` directory.
+For example, we might have a worker name of `pixel3`
+and so under `pixel3/` we will see:
 
 ```
 test.vert                # The default GLSL vertex shader.

docs/glsl-reduce-walkthrough.md

@@ -5,9 +5,15 @@ glsl-reduce is a tool for automatically reducing a GLSL shader with respect to a
 In this walkthrough, we will use a mock shader compiler bug to demonstrate how
 to use glsl-reduce in action.
 
+## Requirements
+
+**Summary:** the latest release zip, Java 8+, and Python 3.5+.
+
+### Release zip
+
 We will be using the latest release zip `graphicsfuzz-1.0.zip`.
 You can download this from the [releases page](glsl-fuzz-releases.md)
-or [build it from source](glsl-fuzz-build.md).
+or [build it from source](glsl-fuzz-develop.md).
 
 Add the following directories to your path:
 
@@ -17,13 +23,15 @@ Add the following directories to your path:
   * `graphicsfuzz-1.0/bin/Mac`
   * `graphicsfuzz-1.0/bin/Windows`
 
-The `graphicsfuzz-1.0/` directory is the unzipped graphicsfuzz release.
+The `graphicsfuzz-1.0/` directory is the unzipped release.
 If building from source, this directory can be found at `graphicsfuzz/target/graphicsfuzz-1.0/`.
 
+### Java 8+
+
 You will also need to install the latest version of the Java 8 Development Kit,
 either:
 
-* From your system's package manager. E.g. `sudo apt-get install openjdk-8-jdk`.
+* From your system's package manager. E.g. Ubuntu: `sudo apt-get install openjdk-8-jdk`.
 * By [downloading and installing Oracle's binary distribution](http://www.oracle.com/technetwork/java/javase/downloads/index.html) (look for Java SE 8uXXX then the JDK link).
 * By downloading and installing some other OpenJDK binary distribution for your platform.
 
@@ -34,16 +42,32 @@ java -version
 # Output: openjdk version "1.8.0_181"
 ```
 
+### Python 3.5+
+
+You will need to install Python 3.5 or higher, either:
+
+* From your system's package manager. E.g. Ubuntu: `sudo apt-get install python3`.
+* By downloading from [https://www.python.org/downloads/](https://www.python.org/downloads/).
+* By downloading and installing some other Python 3 distribution.
+
+For Windows: most recent installers of Python
+add `py`, a Python launcher, to your path.
+Our scripts attempt to use `py -3 SCRIPT.py` to
+execute `SCRIPT.py` using Python 3.5+.
+
+
 ## glsl-reduce in action
 
-We'll start by illustrating a mock shader compiler bug.  We present commands assuming a Linux/Mac environment, but they should be easy to adapt to a Windows setting.
+We'll start by illustrating a mock shader compiler bug.  We present commands assuming a Linux/Mac environment,
+but Windows users can adapt the commands or
+use the Git Bash shell.
 
 ```sh
-# Copy the sample shaders into the current directory:
+# Copy the sample files into the current directory:
 cp -r graphicsfuzz-1.0/examples/glsl-reduce-walkthrough .
 
 # Run the fake shader compiler on the fragment shader file:
-python glsl-reduce-walkthrough/fake_compiler.py glsl-reduce-walkthrough/colorgrid_modulo.frag
+glsl-reduce-walkthrough/fake_compiler glsl-reduce-walkthrough/colorgrid_modulo.frag
 
 # Output:
 # Fatal error: too much indexing.
@@ -53,7 +77,7 @@ python glsl-reduce-walkthrough/fake_compiler.py glsl-reduce-walkthrough/colorgri
 Our fake compiler fails to compile the valid shader because it cannot handle shaders
 with a lot of indexing. Thus, we have found a compiler bug.
 
-> Take a look at `fake_compiler.py` if you want to see why the fake compiler generates this error
+> Take a look at `glsl-reduce-walkthrough/fake_compiler.py` if you want to see why the fake compiler generates this error
 > message for the shader. This is not important for understanding how to use the reducer.
 
 Let's now use glsl-reduce to get a much smaller shader that causes the compiler to fail with this error:
@@ -72,7 +96,7 @@ glsl-reduce glsl-reduce-walkthrough/colorgrid_modulo.json ./glsl-reduce-walkthro
 # <lots of messages about the reducer's progress>
 
 # Confirm that the reduced fragment shader file still reproduces the issue
-python glsl-reduce-walkthrough/fake_compiler.py reduction_results/colorgrid_modulo_reduced_final.frag
+glsl-reduce-walkthrough/fake_compiler reduction_results/colorgrid_modulo_reduced_final.frag
 
 # Output:
 # Fatal error: too much indexing.
@@ -131,7 +155,7 @@ cat slipped_reduction_results/colorgrid_modulo_reduced_final.frag
 # <a shader with an empty main>
 
 # Does it still give us the fatal error?
-python glsl-reduce-walkthrough/fake_compiler.py reduction_results/colorgrid_modulo_reduced_final.frag
+glsl-reduce-walkthrough/fake_compiler slipped_reduction_results/colorgrid_modulo_reduced_final.frag
 
 # Output:
 # Internal error: something went wrong inlining 'floor'.