[FEATURE REQUEST] ✨ use a `BigInt` polyfill for Lynx support

[gauthieramano]

Context

Currently, Lynx doesn't support Convex because its JavaScript Engine (PrimJS) doesn't support BigInt for tricky reasons as we can read here:

• [Feature]: BigInt lynx-family/lynx#941
• Is there any way to use BigInt lynx-family/primjs#37

Request

So, can you consider using a Polyfill for BigInt?
(JSBI's one seems to be a proper one)

Implementation

For instance, this:

// The sign bit is the most significant bit (bit 63)
const v1Sign = (v1Bits & 0x8000000000000000n) !== 0n;

would look like this then:

// outside of the function(s)
const BIT_0 = JSBI.BigInt(0);
const BIT_63 = JSBI.BigInt("0x8000000000000000");

and

// The sign bit is the most significant bit (bit 63)
const v1Sign = JSBI.notEqual(JSBI.bitwiseAnd(v1Bits, BIT_63), BIT_0);

Note

I don't know if there will be extra steps before Lynx fully supports Convex (I hope not), but it seems to be a major (and simple) one.

For instance, I have in mind a potential WebSocket issue, but it seems to be quick & easy to fix:

• [Bug]: WebSocket issue on iOS lynx-family/lynx#951

[thomasballinger]

What does't work today, I would hope that Convex backends can support Lynx by only writing functions that don't require BigInts are arguments or return BigInts?

[gauthieramano]

In short

@thomasballinger each of these 4 expressions from convex-js/src/values/compare.ts breaks a Lynx application at runtime as they are imported in the build:

// Read as BigInt to compare bits
const v1Bits = BigInt(
  new DataView(buffer1).getBigInt64(0, /* little-endian */ true),
);
const v2Bits = BigInt(
  new DataView(buffer2).getBigInt64(0, /* little-endian */ true),
);

// The sign bit is the most significant bit (bit 63)
const v1Sign = (v1Bits & 0x8000000000000000n) !== 0n;
const v2Sign = (v2Bits & 0x8000000000000000n) !== 0n;

My conclusion

I assume that any code imported in a Lynx application shouldn't use anything from the native BigInt object (as arguments, as returned values but also as function implementations, as syntax and as operators).

Here is my process

With these 2 imports in a Lynx application (via TSX files):

import { ConvexReactClient, ConvexProvider } from "convex/react";

and

import { api } from "@packages/backend/convex/_generated/api";

// Note: I'm using Lynx and Convex in a Turborepo project the same way
// React Native & Expo is set in this template:
// https://github.com/get-convex/turbo-expo-nextjs-clerk-convex-monorepo

when I want to run the Lynx application (via npm run dev), there's a Build error exactly at the x character of 0x8000000000000000n at this line:
(this Convex code bellow is included in the bundle of the Lynx application)

// convex-js/src/values/compare.ts > compareNumbers
const v1Sign = (v1Bits & 0x8000000000000000n) !== 0n;

So I assumed the parser (PrimJS) doesn't support the BigInt value syntax.
So, I patched the package locally in order to have:

const v1Sign = (v1Bits & BigInt("0x8000000000000000")) !== BigInt(0);
const v2Sign = (v2Bits & BigInt("0x8000000000000000")) !== BigInt(0);

Then, I had no Build error and I could run the Lynx application on my iPhone, but I instantaneously had this exception:

QuickContext::Execute() exception!!! main-thread.js exception: ReferenceError: BigInt is not defined

(I assume QuickContext is from QuickJS as PrimJS is built on top of it)

with the stack trace starting by:

(react:main-thread)/../../node_modules/convex/dist/cjs/values/values.js
main-thread.js:1979:374

So the JavaScript Engine used by Lynx doesn't support the native BigInt at all.
As we can read in one of the issues I shared in the issue description, @lybvinci wrote:

Currently, the PrimJS version we are using does not enable the CONFIG_BIGNUM macro switch, thus it does not support the BigInt type. Additionally, when using JSC on iOS devices, we need to be compatible with the oldest iOS 9 devices, which also lack support for these APIs. Therefore, we have provided the LynxJSBI tool to partially support some capabilities

So, I assumed that any import in a Lynx application shouldn't use the native BigInt object in order to function. That's why I proposed to use JSBI's polyfill instead, assuming it will do the trick.

The interface of compareNumbers seems right for Lynx, but not its implementation as:

• 0x8000000000000000n and 0n can't be parsed,
• and at runtime, BitInt calls will throw and the & (Bitwise AND) won't certainly have the desired result.

[gauthieramano]

Note

Using JSBI's implementation is great in the sense we can ship 2 builds with the same code:

• one using its BigInt polyfill (which works perfectly with Lynx)
• one using the native BigInt via babel-plugin-transform-jsbi-to-bigint to avoid any performance issue for environments supporting the native BigInt (if there is any notable difference)

Of course, then, we have to figure out a way to use the right build, maybe via specific imports (like convex/lynx instead of convex/react, but about convex/_generated/api, I assume it won't be that trivial)

As the same Convex backend can be used by various applications in a monorepo, I think setting the right version via the convex.json file (with a flag like BigIntPolyfill at false by default) is not the best solution in terms of optimization, but certainly the simplest one (and I assume it would be an acceptable option for those who want their Convex backend to be compatible with Lynx and their web application at the same time).