MobX-State-Tree Performance Testing

This repo is intended to test the performance of MobX-State-Tree. Read a longer form blog post about the intention behind this repository

Dependencies

In order to run the tests here, you'll need:

1. Node and npm in order to run node scripts and download dependencies with npm.
2. bun to run node benchmarks in bun.
3. git for source code management.
4. bash, or anything that can run .sh files
5. Some GNU tools that most Mac OS and Linux systems should have built in: tee, date, grep, tail, awk, sed, echo, rm.

Setup and running tests

npm install # install dependencies
npm run test:node # just test node
npm run test:web # just test web
npm run test:hermes # just test Hermes (not yet supported)

Each one of these commands will write its output to results/<mst-version>-<timestamp>-<platform>-results.csv. Each scenario has a title, which should provide a short description of what it demonstrates. We use Benchmark.js to get the operations per second of that scenario, along with the margin of error and number of runs used to reach statistical significance. The last column of the result is the maximum memory usage during testing for that scenario, in bytes.

Changing MST versions

If you want to run the suite against different versions of MST, before you run tests, you can:

npm install mobx-state-tree@desired-version --save

And your output file will write the version number in the file name to make it easy to keep track of.

How to add scenarios

If you'd like to measure some kind of operation in MobX-State-Tree, you'll need to add a scenario to scenarios/index.js. Every scenario must be a plain JavaScript object with:

1. title: this is the title you'll see during each test run, and in the results
2. longDescription: not yet used, but please consider writing more detail about your scenario.
3. run: this should be a synchronous function that exercises what you want to test.

Each scenario must be exported from scenarios/index.js to be picked up. Here's an example:

import { createNModels } from "./model-creation.js";

export const scenario1 = {
  title: "Create 1 model",
  longDescription:
    "Create 1 model with all primitive types, along with a setter action for each.",
  run: () => {
    createNModels(1);
  },
};

These exported functions get imported in ./runner.js, and then bundled for node and web separately. We run the node bundle with node, and we have a Puppeteer script that loads, executes, and reports the output of the web bundle. That's at puppeteer.cjs.

Feel free to write helper functions and different types of set up in the scenarios folder. Please avoid using APIs specific to web browsers, node, or React Native. We aren't yet set up to handle those differences.