Testing: create docker container for interactive testing of specific browser versions

[dhrogers]

We need a clean testing environment for supported version testing of Chrome, Firefox and Safari. This will help with cross-browser testing, and regression testing of releases.

If there is an automated way to test browser versions, that would be preferred, but a simpler first step could be a clean docker container with specific versions of browsers.

Unfortunately, we would need to do the cross-product of OS and browser, so:

• OS: Linux, OSX, WIndows
• Browser: Chrom, Firefox, Safari

Other options welcome ...

[camtauxe]

Unfortunately, Docker really isn't meant to be used with GUI applications, though it can kinda-sorta be done. Looks like there are containers for Firefox and Chrome HERE and HERE (though the Chrome one seems heavily tied to some other enterprise thing). These allow you to open the browser either through VNC or inside another browser.

Bear in mind that Docker containers are also Linux-only, so testing different OS's or Safari won't be possible. For true cross-product testing, we would probably need to use VMs, which gets much more complicated. Cross-browser testing like this (especially with different versions of browsers) is pretty complex, so we may want to instead invest in an enterprise solution. Lambdatest and Browserstack are the first two that popped up when I started googling. Those will handle the infrastructure issues of configuring, spinning up and connecting two VMs/emulators, but of course cost big $$$.

By the looks of it, both of those products are based off of Selenium Grid (you'll remember Selenium from last time we brought up testing). That's free, but it doesn't look like something we can host on our own, and setting it up looks very hard.

It's not interactive, but if it's just browser compatibility we're worried about. We could use a package like eslint-plugin-compat, which can scan our Javascript and detect browser compatibility issues. It's a NodeJS package, so it would be relatively easy to integrate into our current test suite (at least compared to these other options).

Also, as a disclaimer, I should stress that this is all just based off about 30 minutes of Googling. I don't have any personal experience with these kinds of testing workflows (other than using Docker, in general, of course), so my assessment on how easy or difficult any of this may be off-base. In addition, several of the sources I found regarding interactive cross-browser testing came from the blogs of the very same companies I mentioned above. So there's a definite bias as the articles made sure to mention that, of course, the best solution was to buy their product 😉