Wasm: Performance problems in Firefox

[xermicus]

It was observed that Firefox fails to compile contracts that compile fine in other Browsers. Likely due to memory pressure

[smiasojed]

Firefox memory stats for compilation with web worker:
Total Memory Usage
17,193.75 MB (100%) – Total explicit memory usage.
Web Workers (91.33% of total)
15,703.66 MB (91.33%) is consumed by Web Workers.

1. Main Worker - compiler (78.61% of total) libs_remix-solidity_src_lib_es-web-worker_compiler-worker_ts.0.54.0-dev.1738580472454.js
   13,515.60 MB (78.61%) is used by the worker
   |__12,100.63 MB (70.38%) – WASM memory (non-heap/elements/wasm).
   |__1,312.73 MB (7.63%) – WebAssembly.Instance objects.
   |____1,011.13 MB (5.88%) – Compiled WebAssembly code (non-heap/code/wasm).
   |____301.60 MB (01.75%) -- malloc-heap
2. Secondary Worker (12.69% of total) libs_remix-solidity_src_lib_es-web-worker_compiler-worker_ts.0.54.0-dev.1738580472454.js
   2,182.16 MB (12.69%) is used by the worker.
   |__ 1,798.13 MB (10.46%) WASM memory (non-heap/elements/wasm).
   |__ 316.70 MB (0.1%) WebAssembly.Instance objects.

[smiasojed]

To check:

1. why do we have two instances of the compiler worker
2. can we do something with the linear memory (ArrayBuffer) for Wasm allocations

[athei]

why we have to instances of the compiler worker

Yeah the seem to be based on the same javascript file. So they are instances of the same thing. 14GB of RAM usage for the first one seems to be a bit extreme. Maybe, just maybe we are lucky and just something went wrong with the first one and the second one does the right thing. 2GB of RAM seems more realistic.

can we do something with the linear memory (ArrayBuffer) for Wasm allocations

Yeah that seems to be the big chunk: WASM memory (non-heap/elements/wasm). I guess it refers to linear memory and elements (indirect function calls). But linear memory is basically everything. Like there is no other memory available to the program. But I don't know what malloc-heap is then.

[smiasojed]

The second one is not used,. The allocations of the first one are changing during compilation.

[athei]

Remember how bun runs into some allocation error for the element table? Maybe it is related? Like it is not actual memory allocations but something goes wrong with the element table? As it seems to be the same memory area. Just a hunch.

Because it seems very unlikely that the compilation would make so many allocations. We are not compiling a huge code base. Thats usually only of you lto a huge code base.

Could also be a memory leak in a tight loop.

[athei]

Can you compare memory usage to Chrome? Would be interesting if it uses less RAM on Chrome or if Chrome just allows more RAM.

[smiasojed]

In the case of Chrome, we have also two compilation workers, each consuming 2.1GB of memory. Out of this, 1.9GB is allocated to WASM Memory (ArrayBuffer).

In Firefox, the first compilation always seems to pass, resulting in memory consumption similar to Chrome. However, on the next compilation, memory usage increases to 13.5GB for the compilation worker, with 12.1GB allocated to the ArrayBuffers.
Forcing garbage collection (GC) and cycle collection (CC) appears to release unused memory and caches, bringing it back down to 2GB, allowing us to successfully perform another compilation.

We are building revive with default WASM mem limit set to 2GB:
from emscripten doc: Note that the default value here is 2GB, which means that by default if you enable memory growth then we can grow up to 2GB but no higher., what suggests that there is something wrong with Revive execution in Firefox, probably old Wasm instances have not been deallocated.

[xermicus]

I think seeing two workers is expected because resolc calls itself recursively and I'm not aware that we changed this architecture for in-browser environments?

[smiasojed]

I think seeing two workers is expected because resolc calls itself recursively and I'm not aware that we changed this architecture for in-browser environments?

We are running resolc in a worker, so if resolc makes a recursive call, it is executed within this worker. The original Remix code also has two workers with soljson compiler.

[athei]

So tl;dr: The memory usage balloons only on Firefox? Because 12GB of mem usage would also not go well with Chrome I suppose.

[smiasojed]

yes, on Firefox only.
On Chrome it is stable 2GB:

In the case of Chrome, we have also two compilation workers, each consuming 2.1GB of memory. Out of this, 1.9GB is allocated to WASM Memory (ArrayBuffer).