Jest incompatibility: Dynamic Import Callback Error

[blakewest]

Hi, I was trying to introduce PG into my codebase, and needed a way to run my tests against it. I'm using Prisma, and saw this and thought the adapter looked great. However, it seems there's some compatibility issue with Jest. If I try to import it, I get an error that breaks everything. See below for an example stack trace.

rebalance-batches-store › completeSimulations › completes

    PrismaClientKnownRequestError: 
    Invalid `prisma.rebalanceBatch.create()` invocation in
    /Users/blakewest/code/warbler-labs/mono/packages/mesh/tests/models/rebalance-batches-model-pg.test.ts:24:35

      21 const now = new Date();
      22 // Create 20 batches, with batch 0 being the most recent
      23 for (let i = 0; i < 20; i++) {
    → 24   await prisma.rebalanceBatch.create(
    A dynamic import callback was invoked without --experimental-vm-modules

      22 |     // Create 20 batches, with batch 0 being the most recent
      23 |     for (let i = 0; i < 20; i++) {
    > 24 |       await prisma.rebalanceBatch.create({
         |       ^
      25 |         data: {
      26 |           id: i.toString(),
      27 |           createdAt: new Date(now.getTime() - i * Duration.ofSeconds(1).toMillis()),

      at _n.handleRequestError (../../node_modules/@prisma/client/runtime/library.js:121:7749)
      at _n.handleAndLogRequestError (../../node_modules/@prisma/client/runtime/library.js:121:7057)
      at _n.request (../../node_modules/@prisma/client/runtime/library.js:121:6741)
      at async l (../../node_modules/@prisma/client/runtime/library.js:130:9355)
      at async Object.<anonymous> (tests/models/rebalance-batches-model-pg.test.ts:24:7)

In my investigation, I saw that 1.) Jest seems to have had a similar issue last year with Prettier V3. They said it would be fixed in Jest 30, but I installed the latest (alpha) release of Jest 30, and the same error occurs. , and 2.) If I removed the PG lite adapter, the tests work fine.

I'm not actually 100% if this is a Jest issue or a ElectricSQL issue, but I thought I'd raise it.

Has anyone else been able to use this PG-lite package with Jest 29?

[samwillis]

This will need some investigation to solve, a few initial thoughts / pointers:

• I believe Jest was CJS only until recently, we do have both ESM and CJS builds in the in the package, but the CJS build is newer and less exercised.
• There are a few places in the codebase where we branch depending on environment being browser/node, on the node branches a dynamic import is used to pull in additional dependencies. We have had issues with bundlers with this and it could be similar.
• We use import.meta and __dirname to locate the WASM binaries, Jest may not be happy with this.

Writing a library that supports both node and the browser is complex and prone to these issues unfortunately, add in WASM and it gets worse. One day we will all have moved to ESM and the tooling will be perfect.

I'm very keen to find the fix for this as Jest is so widely used.

[firelizzard18]

In vscode I can add the following to settings:

"jest.nodeEnv": {
        "NODE_OPTIONS": "--experimental-vm-modules"
}

However now I get:

TypeError: The URL must be of scheme file
    at open (node:internal/fs/promises:638:10)
    at Module.readFile (node:internal/fs/promises:1254:20)
    at ri (/home/firelizzard/src/athletifi/website/node_modules/@electric-sql/pglite/src/utils.ts:73:16)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

presumably in the code that's using import.meta to locate the WASM binaries.

[saenzramiro]

Having PGLite running in Jest test suites will be wonderful.

[firelizzard18]

The import.meta.url error was caused in part by @jest-environment: jsdom. The bundled version of pglite attempts to guess at what import.meta.url should return, and if document exists it assumes it's being run in a browser and returns http://localhost/main.js. By setting the Jest environment to node I was able to get past this. However, I ran into another issue:

pglite/packages/pglite/src/postgresMod.ts

Line 1 in 75f9f6d

import PostgresModFactory from '../release/postgres.js'

Because the explicit .js, Jest throws an error when release/postgres is not an ESM module. I fixed this simply by removing .js. I opened #423 to make this change.

[vitonsky]

For those who just start use pglite and faced with this problem that make not possible to implement tests on jest for code that uses pglite, hey there i'm with you.

In my project nano-queries that implements database-agnostic SQL query builder i faced with this problem too.

By the nature of project purpose, i critically need to implement tests with real world cases for some target databases like PGlite. So it would be nice to resolve current issue. I did not found any other query builders that would be enough abstract to build queries for PGlite, this is why i think my project is direct contribution to PGlite infrastructure. So i ask a core team of PGlite to prioritize current issue, since it is quite important for end users. My opinion is that problems with DB testing may block community members of implement solutions around PGlite.

Temporary workaround

If you critically needed in tests for code that uses PGlite and you use Jest, you may try to move to vitest or at least move to vitest partially, to run those tests that you can't run with Jest.

This is how i solved this problem. Vitest have back compitability with Jest, so for most cases you will not have to make changes in tests, but just install vitest, enable global hooks and run tests with vitest.

But addressing to the pglite team, keep in mind that problem with Jest still is important, since there are a many reasons why some projects can't migrate to another testing framework. Thank you.

[firelizzard18]

@vitonsky Have you tried running with NODE_OPTIONS=--experimental-vm-modules and setting the jest environment to node? I currently have working jest tests that use pglite.

[vitonsky]

@vitonsky Have you tried running with NODE_OPTIONS=--experimental-vm-modules and setting the jest environment to node? I currently have working jest tests that use pglite.

Sure. I had another problems one by one after this change. I use ts-jest with jest, since my tests written in typescript, and next error i remember was ReferenceError: exports is not defined that has not been solved with any change.

If you need to debug a problem, you may check repository on this commit https://github.com/vitonsky/nano-queries/tree/a4057fec092e756bc6c50bb2c80c6c06247f28fb

Just try to install pglite, add and then run trivial test like this

import { PGlite } from '@electric-sql/pglite';

test('test', () => {
  const db = new PGlite();
  test(1).toBe(1);
});

I use node v22.12.0 installed with nvm

[firelizzard18]

I'm not really a contributor to this project, I just submitted #423 to make a tiny change to fix it for myself. I'm just curious because our scenarios seem quite similar. Maybe something about Next.js is making it work for some reason. I can't share my code but I've attached some config files in case they're helpful. Most of the tests use the jsdom environment, so the tests that need pglite set that on a per-file basis. config.zip