RFC: test-runner plugin to intercept modules

[tmsns]

Hi there! 👋

Problem statement

I've been creating tests in JavaScript for quite some time. A practice I see coming up every so often, especially when creating unit tests, is hooking into dependencies and overriding the default behaviour of used functions. When these dependencies are being accessed from the global window object, it is pretty easy to override the behaviour of functions (eg. window.fetch())

Nowadays, however, we live in an ES modules world, where dependencies are brought in using an import of the module. When creating tests for such systems it is not always easy to override the behaviour of imported modules, as normal exported functions are not allowed to be overridden. There have been a number of solutions until now:

• adding hooks into your own code to override the dependencies, eg.

import { helper } from 'some-library';
export function myFunction(a, b, _helper = helper) {
  ...
  _helper();
  ...
}

These types of solutions clutter your code, and are not always easy to create, especially when the system under test is a class or a component.

• using import maps
  Although import maps deliver the ability to intercept modules, their usage within tests is not really straightforward. Also, overriding behaviour multiple times, requires a more elaborate setup.

Proposal

With @web/test-runner and @web/dev-server, we embrace the usage of ES modules and other standard practices. But we also get a powerful system to customize the experience.

This allows us to create a plugin that enables us to actually intercept import calls, and change module exports when running tests in @web/test-runner: https://github.com/tmsns/web/tree/intercept/packages/test-runner-intercept

Here is a sample of the proposed usage of the plugin. The idea is to keep its usage as simple and straightforward as possible (note: I've already been doing some back-and-forth on this with @LarsDenBakker)

Setup:

// web-test-runner.config.mjs
import { interceptModulePlugin } from '@web/test-runner-intercept/plugin';

export default {
  plugins: [interceptModulePlugin()],
};

// src/getTimeOfDay.js
import { getCurrentHour } from 'time-library';

export function getTimeOfDay() {
  const hour = getCurrentHour();
  if (hour < 6 || hour > 18) {
    return 'night';
  }
  return 'day';
}

Simple test scenario:

// test/getTimeOfDay.test.js
import { interceptModule } from '@web/test-runner-intercept';

const timeLibrary = await interceptModule('time-library');
const { getTimeOfDay } = await import('../src/getTimeOfDay.js');

describe('getTimeOfDay', () => {
    it('returns night at 2', () => {
        timeLibrary.getCurrentHour = () => 2;
        const result = getTimeOfDay();
        if (result !== 'night') {
            throw;
        }
    });
});

More extended test scenario with common helper libraries:

// test/getTimeOfDay.test.js
import { stub } from 'sinon';
import { expect } from 'chai';
import { interceptModule } from '@web/test-runner-intercept';

const timeLibrary = await interceptModule('time-library');
const { getTimeOfDay } = await import('../src/getTimeOfDay.js');

describe('getTimeOfDay', () => {
  it('returns night at 2', () => {
    const stubGetCurrentHour = stub(timeLibrary, 'getCurrentHour').returns(2);
    try {
      const result = getTimeOfDay();
      expect(result).to.equal('night');
    } finally {
      stubGetCurrentHour.restore();
    }
  });
  it('returns day at 14', () => {
    const stubGetCurrentHour = stub(timeLibrary, 'getCurrentHour').returns(14);
    try {
      const result = getTimeOfDay();
      expect(result).to.equal('day');
    } finally {
      stubGetCurrentHour.restore();
    }
  });
});

As this functionality is so useful when creating tests with ES modules, the hope is that this plugin could be an official plugin of the @web monorepo and maybe even bundled in @web/test-runner. 🙈

What does the community think?

• Do you think it would be useful to have this functionality?
• Is the syntax / usage straightforward and easy to understand?
• How should it be integrated in @web/test-runner?

The repo with the plugin is a barebones project, but the code does actually work. So, if you already have some questions or remarks on the implementation, feel free to share this as well. Of course, if we go further with this, we'll need to add tests, document it, and convert it to TS. But for now, I kept it lightweight.

[bashmish]

Can't you do this without a plugin? Some edge cases that don't work?

From what I know, an ES module is a global singleton, so if you import it via a normal static import syntax like import * as timeLibrary from 'time-library', then you can stub it's exports as well.

[tmsns]

Well, that's actually the "limitation" this plugin tries to workaround. ES modules don't allow the modification of named imports or members of a namespace import.

When trying to re-assign a named import, the user agent throws with:

import { getCurrentHour } from 'time-library';

getCurrentHour = () => 5; // Uncaught TypeError: Assignment to constant variable.

When trying to re-assign a member of a namespace import:

import *  as timeLibrary from 'time-library';

timeLibrary.getCurrentHour = () => 5; // Uncaught TypeError: Cannot assign to read only property 'getCurrentHour' of object '[object Module]'

More info on this designed "limitation":

• https://developer.mozilla.org/en-US/docs/web/javascript/reference/statements/import#imported_values_can_only_be_modified_by_the_exporter
• https://eslint.org/docs/latest/rules/no-import-assign
• Case for module mocking for testing WICG/import-maps#131

[bashmish]

interesting, what kind of magic did I have in my recent test setup that allowed me to do this kind of stuff without running into this exception :D

but you are right, good proposal!

[trusktr]

• using import maps

This is the way to go. It will make code highly-standard and increase its longevity. Anything else reduces simplicity of test-runner, will be more maintenance work, and could mean breaking changes and breakages down the road, and more difficult to port to other systems later.

Although import maps deliver the ability to intercept modules, their usage within tests is not really straightforward. Also, overriding behaviour multiple times, requires a more elaborate setup.

What issues did you have with import maps? Have you already tried importMapsPlugin? It is fairly straight forward to dynamically define import maps (intercept imports) for matched files.

The only difference is that for some test, the interception is not located in the test file, but in the test-runner config, which is not too big of a deal at the end of the day. You can comment in the test file like so:

import foo from 'foo' // mocked in test-runner import map
import bar from 'bar' // mocked in test-runner import map
import baz from 'baz'

then you know where to look. Then you don't need top-level await import()

Basically it seems that adding the complexity is not worth just to define the module mock inside the test file.

[tmsns]

Hi Joe,

First off, thanks for reacting! 😃

I agree, import maps would be a great way to intercept modules for mocking. My first trials were actually using import maps (instead of a resolution hook). I was able to intercept modules just fine, and was using scopes to make sure to be able to retrieve the original implementation.

<script type="importmap">
  {
     "scopes": {
       "/src/": {
         "time-library": "./test/time-library.mock.js"
       }
     }
  }
</script>

// ./test/time-library.mock.js
import { getCurrentHour as _getCurrentHour} from 'time-library';

export function getCurrentHour() {
  return (_getCurrentHour() + 12) % 24; // dummy mock that moves time with 12 hours
}

And indeed, in this case, the interception is defined in the config (using a custom plugin, or the importMapsPlugin) in order to have the import map injected in the HTML. So all was going fine.

The problems arise when scaling up this solution in a real life project with many tests. And the main cause for them is the static ./test/time-library.mock.js file. Some of the problems encountered:

• It's not possible to change the implementation of the mocked getCurrentHour() between multiple tests in the same test file. This is a big one. It would mean people need to split all their tests in different test files in order to use a specific mock in their test.
• Having the mocked function in a separate file next to the test file is quite inconvenient. Usually when mocking, mocks are added during the arrange stage of a test (eg. https://medium.com/@pjbgf/title-testing-code-ocd-and-the-aaa-pattern-df453975ab80) Having another way to do this, will throw off a lot of devs, as they are used to doing it within the test (using the AAA pattern or different)
• As the complete module is intercepted, all other used methods of the library need to be mocked or rewired as well. This makes creating these static stubs a lot more cumbersome. There is more boilerplate code that needs to be created and maintained by hand. This seems trivial, but a lot of devs are not looking forward to creating (ormaintaining) tests. So, making it easy to create mocks, is worth it, as it will make the maintenance of tests a lot easier.

Now, this being said, most of these problems can be solved by using import maps in the backend of a plugin (as the module intercepting technology), having the mock files being generated automatically and having syntactic sugar within the test (like in the proposal) to make intercepting modules more straightforward. There is a downside however. Import maps cannot be added/changed on a page during module execution. This means that a user would need to add additional plugin config to define (upfront) what modules need to be intercepted for each test file. This adds complexity in the setup of tests.
This is on top of the additional complexity of using import maps in the plugin backend:

• the plugin needs to transform the html of the test run to add the import map (similar to the importMapsPlugin) to point to dynamic mock files on new (dynamic) urls
• the plugin needs to serve those dynamically created mock files on the new dynamic urls

This is why the proposal is using the resolution hook in order to simplify the implementation of the interception. As an added benefit it solves the need of defining upfront (in an additional config) what modules need to be intercepted: additional resolution hooks can be defined during a test run.
It would be great to switch back to import maps once dynamic import maps get supported (https://github.com/WICG/import-maps#programmatic-api). In the mean time resolution hooks are pretty common in backend JS systems:

• https://nodejs.org/docs/latest-v21.x/api/module.html#resolvespecifier-context-nextresolve
• https://rollupjs.org/plugin-development/#resolveid
• https://webpack.js.org/api/resolvers/

So, porting this to other systems should be pretty straightforward, if needed. 😃

I hope this gives more insight as to the choice of a resolution hook was made.

[trusktr]

Ah, I see! Thanks for painting a more detailed picture. That makes sense. I suppose my tests haven't gotten complex enough for me to feel those pain points yet!

Then what you've made looks very nifty from a usage perspective.

Could this be implemented in vanilla ES Modules somehow, with help of a service worker? What I'm wondering is, if Module objects returned from import() are not writable, is it even possible? Does wtr/puppeteer have special abilities to patch modules that we otherwise don't have in a normal browser env?

[tmsns]

Could this be implemented in vanilla ES Modules somehow

Yes, the complete proposal is implemented using vanilla ES Modules (the test runner plugin, the code of the module facade, and the test helper that retrieves the mutable object)

with help of a service worker

Briefly looked at it some time ago, but saw a couple of hurdles:

• Only one service worker can be installed per path. With the introduction of @web/mocks (using MSW), a service worker will already be active while testing. It felt complex to start juggling between multiple service workers.
• The interception would work differently. Instead of changing the initial resolution (allowing the original module url to be accessed), the service worker will probably need to add the proxy code as part of the original module (ie. patching it). This could create code conflicts.
• Managing the lifecycle of a service worker would require extra code to make sure it is correctly cleaned up

But as such, just like import maps, if service workers seem to be a better fit, the resolution hook technique could be replaced by a service worker. What do you think, @thepassle?

What I'm wondering is, if Module objects returned from import() are not writable, is it even possible

It's possible by using the same technique as the current proposal: having a special named export that exposes a (proxy) object that can override the exports. The issue is that it will probably need to be added to the original module (as opposed to the proposal where we change the resolution to a new dynamically created module)

Does wtr/puppeteer have special abilities to patch modules that we otherwise don't have in a normal browser env?

Using the resolution hook, it is possible to patch the modules that require the import. So, in the proposal, we actually leave the module to intercept alone. And this is not available in a normal browser afaik, except with import maps.

[thepassle]

Im not sure if doing this in a service worker actually benefits anything, so im fine doing it in a plugin.

We can make the mock service worker combined with logic from another service worker, but it would indeed be more complex, because you'd need cross-plugin communication to combine the logic of plugins that use a service worker.

Service worker lifecycle shouldnt be a problem, as evidenced by usage of msw via @web/mocks

[tmsns]

As people seem generally positive on this proposal, I will continue by creating a full-fletged PR (tests, docs, examples,, ..) where the plugin is delivered in a sub project for the web monorepo.

Thanks already for the comments!

[tmsns]

A draft PR has been created #2546. You can already check out the code there. Tests and readme still need to be updated.