[Question]: How to cache on github actions?

[YonatanKra]

Hi,
I've setup github actions on Linux and it works fine.
I'm now running the same tests on MacOS and the binaries are not the same, so caching the installation is not working (for obvious reasons).
What is the recommended way to cache the playwright binaries for MacOS on github actions?

[mxschmitt]

General recommendation is to not cache them, since they are downloaded in less than 10 seconds. If you are still want to cache them, you find the locations here. The cache key should contain the OS and Playwright version.

[YonatanKra]

It's taking ages to install playwright.
Around 8 minutes (on github actions MacOS machines).
So caching is the easiest solution I could think of - but it doesn't work.
I've cached these libraries and it still doesn't work.
Here's my code:

     - name: Cache playwright binaries
        uses: actions/cache@v2
        id: playwright-cache
        with:
          path: |
            ~/Library/Caches/ms-playwright
          key: vivid-cache-playwright

     - name: Install playwright deps
        if: steps.playwright-cache.outputs.cache-hit != 'true'
        run: yarn add playwright -D -W && yarn playwright install-deps

But on the actual tests run step, I get the following error: browserType.launch: Failed to launch webkit because executable doesn't exist at /Users/runner/Library/Caches/ms-playwright/webkit-1446/pw_run.sh.

When I list the folder, I can actually see the files but with webkit-1442.

Is there a way to install a certain webkit version?

[mxschmitt]

Are you using self-hosted runners? If this problem is persistent on the hosted-runners, it might make sense to contact the GitHub Actions support. We run 50-100 jobs daily and didn't experience such big delays yet.

Also you are using a private NPM Registry, which might add delay. But the browsers are not getting downloaded over NPM, thats a normal http request then.

Running npx playwright install will automatically install the right version for you.

[YonatanKra]

We are using the gha runners. We have no self hosted stuff. The strange thing is - the Mac machines run consistently slower in everything. The tests run roughly 50% longer and running yarn install takes around 300% longer (compared to similar jobs on Linux).

Anyway, I've managed to solve the issue.

On the Mac machine running the tests I've added this code:

     - name: Cache playwright binaries
        uses: actions/cache@v2
        id: playwright-cache
        with:
          path: |
            ~/Library/Caches/ms-playwright
            **/node_modules/playwright
          key: cache-playwright-macos

      - name: Install playwright deps
        if: steps.playwright-cache.outputs.cache-hit != 'true'
        run: yarn add playwright -D -W

Now it has a cache for playwright only. I runs a lot faster than the actual yarn add playwright.
Before the change:

After the change:

I will check what might be the issue with gha on mac for us as we thought it is a common issue for everyone. Thanks a lot!

[aslushnikov]

@YonatanKra 8 minutes! This is too much! We ourselves run installation on GHA-based macs, and it's never that bad. The whole npm ci for Playwright itself takes 1m including browser downloads.

Is it a public repo? Would you mind sharing a link to the GHA action so that I can look into logs?

[mxschmitt]

Closing since the issue seems stale as part of the triage process. Please create a new issue if you still face any issues. 8 Minutes is definitely not as-designed but sounds more a infra issue.

[YonatanKra]

@YonatanKra 8 minutes! This is too much! We ourselves run installation on GHA-based macs, and it's never that bad. The whole npm ci for Playwright itself takes 1m including browser downloads.

Is it a public repo? Would you mind sharing a link to the GHA action so that I can look into logs?

Hi,
Sorry for the late reply. It's a private repo, so I cannot share a link.
The long time might be due to jfrog artifactory. I'll check it out this week.

[jesstelford]

I extended the above caching logic to support new browser binaries as the version of Playwright changes. This works for yarn, but an equivalent should be possible with npm.

      - name: Install dependencies
        run: yarn

      # Figures out the version of playwright that's installed.
      # 1. Because we don't know what version yarn will resolve it to, we have
      #    to use `yarn why` to get the actually installed version.
      # 2. Because we're in a workspace, we need to make sure we get the version
      #    for the root and not any children, hence the `grep`. If not using
      #    workspaces, this can be skipped.
      # 3. jq comes pre-installed in the Ubuntu runner, so we use that to get
      #    the correct version string.
      # 4. Finally, we use sed to extract just the version number (eg; '1.22.0')
      # The result is stored in steps.playwright-version.outputs.version
      - name: Get installed Playwright version
        id: playwright-version
        run: echo "::set-output name=version::$(yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://')"

      # Attempt to restore the correct Playwright browser binaries based on the
      # currently installed version of Playwright (The browser binary versions
      # may change with Playwright versions).
      # Note: Playwright's cache directory is hard coded because that's what it
      # says to do in the docs. There doesn't appear to be a command that prints
      # it out for us.
      - uses: actions/cache@v3
        id: playwright-cache
        with:
          path: '~/.cache/ms-playwright'
          key: '${{ runner.os }}-playwright-${{ steps.playwright-version.outputs.version }}'
          # As a fallback, if the Playwright version has changed, try use the
          # most recently cached version. There's a good chance that at least one
          # of the browser binary versions haven't been updated, so Playwright can
          # skip installing that in the next step.
          # Note: When falling back to an old cache, `cache-hit` (used below)
          # will be `false`. This allows us to restore the potentially out of
          # date cache, but still let Playwright decide if it needs to download
          # new binaries or not.
          restore-keys: |
            ${{ runner.os }}-playwright-

      # If the Playwright browser binaries weren't able to be restored, we tell
      # paywright to install everything for us.
      - name: Install Playwright's dependencies
        if: steps.playwright-cache.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps

[birtles]

It might be my version of yarn but yarn --why didn't produce anything with children in it for me so I ended up parsing yarn.lock instead:

echo "::set-output name=version::$(cat yarn.lock | grep "^\"@playwright\/test" --after-context 1 | tail -n 1 | sed 's/\s*version "\([0-9.]\+\)"/\1/')"

Furthermore, I found that while the above caches the downloaded versions, the playwright runtime dependencies still need to be installed. Hence, I ended up with the following for the last step:

      - name: Install Playwright with dependencies
        if: steps.playwright-cache.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps

      - name: Install Playwright's dependencies
        if: steps.playwright-cache.outputs.cache-hit == 'true'
        run: npx playwright install-deps

[NikolayMakhonin]

yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://')

This is crossplatform version of this:

node -e "console.log(require('playwright/package.json').version)"

[jesstelford]

@NikolayMakhonin that only works if playwright and @playwright/test are version locked, but I don't think they are.

[andykenward]

yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://')

This is crossplatform version of this:

node -e "console.log(require('playwright/package.json').version)"

I think we would need to use npm ls @playwright/test or npm ls @playwright/test --json to get the version that's actually installed. Can someone help with how we can parse the output for the version number?

npm ls @playwright/test --json

{
  "version": "0.1.0",
  "name": "your-project-name",
  "dependencies": {
    "@playwright/test": {
      "version": "1.24.0",
      "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.24.0.tgz"
    }
  }
}

[chrisvariety]

@andykenward echo -n "::set-output name=version::$(npm ls @playwright/test --json | jq --raw-output '.dependencies["@playwright/test"].version')"