Discussion: making the result table more compact

[pallosp]

My workflow includes running benchmarks in the VS Code terminal regularly.

Currently I can't give the tasks longer names than 20 characters, because they would make the table overflow and break its layout. Therefore I'm suggesting a couple of changes to make the metric columns narrower.

(index)

• Rename the column to #. Saves 5-6 characters.

Latency average

• Limit its precision. The digits after the 5th one are just noise. While this won't save more than 1-2 characters, it will make the numbers easier to read.
• Optional: rename average to avg.

Latency median

• Rename median to 50%.
• Drop the ± part from the numbers. I don't think they give any actionable insight.
• Drop the trailing zeros. Round to integer when the median has too many digits.
• Return numbers, not strings.

These changes together may save 3-7 characters.

Throughput average

• Optional: rename average to avg.

Throughput median

• Rename median to 50%. Saves 3 characters.
• Return numbers, not strings.

Before

Inserting 50k elements into various data structures
┌─────────┬──────────────────────┬──────────────────────┬───────────────────────┬────────────────────────────┬───────────────────────────┬─────────┐
│ (index) │ Task name            │ Latency average (ns) │ Latency median (ns)   │ Throughput average (ops/s) │ Throughput median (ops/s) │ Samples │
├─────────┼──────────────────────┼──────────────────────┼───────────────────────┼────────────────────────────┼───────────────────────────┼─────────┤
│ 0       │ 'float keyed Map'    │ '3218139.39 ± 3.66%' │ '2756229.00 ± 146.00' │ '331 ± 2.19%'              │ '363'                     │ 312     │
│ 1       │ 'int keyed Map'      │ '2211478.20 ± 1.77%' │ '2076166.00'          │ '462 ± 1.06%'              │ '482'                     │ 453     │
│ 2       │ '10 int keyed Maps'  │ '1304029.77 ± 1.12%' │ '1218542.00'          │ '782 ± 0.88%'              │ '821'                     │ 767     │
│ 3       │ 'sparse Array, 0.5'  │ '147581.90 ± 1.37%'  │ '124250.00'           │ '7543 ± 0.44%'             │ '8048'                    │ 6776    │
│ 4       │ 'sparse Array, 0.1'  │ '655357.51 ± 0.44%'  │ '654250.00'           │ '1537 ± 0.43%'             │ '1528'                    │ 1526    │
│ 5       │ 'sparse Array, 0.05' │ '1293428.82 ± 0.42%' │ '1306833.50 ± 0.50'   │ '776 ± 0.51%'              │ '765'                     │ 774     │
└─────────┴──────────────────────┴──────────────────────┴───────────────────────┴────────────────────────────┴───────────────────────────┴─────────┘

After

Inserting 50k elements into various data structures
┌───┬──────────────────────┬──────────────────────┬──────────────────┬────────────────────────────┬────────────────────────┬─────────┐
│ # │ Task name            │ Latency average (ns) │ Latency 50% (ns) │ Throughput average (ops/s) │ Throughput 50% (ops/s) │ Samples │
├───┼──────────────────────┼──────────────────────┼──────────────────┼────────────────────────────┼────────────────────────┼─────────┤
│ 0 │ 'float keyed Map'    │ '3218139 ± 3.66%'    │ 2756229          │ '331 ± 2.19%'              │ 363                    │ 312     │
│ 1 │ 'int keyed Map'      │ '2211478 ± 1.77%'    │ 2076166          │ '462 ± 1.06%'              │ 482                    │ 453     │
│ 2 │ '10 int keyed Maps'  │ '1304029 ± 1.12%'    │ 1218542          │ '782 ± 0.88%'              │ 821                    │ 767     │
│ 3 │ 'sparse Array, 0.5'  │ '147581 ± 1.37%'     │ 124250           │ '7543 ± 0.44%'             │ 8048                   │ 6776    │
│ 4 │ 'sparse Array, 0.1'  │ '655357 ± 0.44%'     │ 654250           │ '1537 ± 0.43%'             │ 1528                   │ 1526    │
│ 5 │ 'sparse Array, 0.05' │ '1293428 ± 0.42%'    │ 1306833          │ '776 ± 0.51%'              │ 765                    │ 774     │
└───┴──────────────────────┴──────────────────────┴──────────────────┴────────────────────────────┴────────────────────────┴─────────┘

If you agree, I'm happy to send a pull request.

[jerome-benoit]

You can use a custom convert arrow function (https://tinylibs.github.io/tinybench/classes/Bench.html#table) to format the results the way you want.
What is currently missing is example usage of it.

[pallosp]

Thanks, that's a great workaround.

I think there is still value in improving the default rendering, as it reduces both the time to write benchmarks and the time to interpret the results.

[jerome-benoit]

The main issue is that same words are repeated because console.table() is not supporting columns grouping.

You can make a PR to refine the default, but:

• median is more explicit than 50% alone. To make it explicit, the term percentile should be added. med is a fine shortcut
• the standard deviation is a meaningful information but should only be displayed if upper than a threshold (8%)

[pallosp]

It turns out that console.table() is pretty dumb. It doesn't even allow to remove to rename the index column.

Are you suggesting to rename average to avg and median to med?

The standard deviations are indeed useful. I didn't want to remove them. What I found less useful is the distance between the two middle samples when there are even number of them.

[jerome-benoit]

Are you suggesting to rename average to avg and median to med?

It was your suggestions ;-) I just think that 50% is less self-explanatory than med.

The standard deviations are indeed useful. I didn't want to remove them. What I found less useful is the distance between the two middle samples when there are even number of them.

It's in fact the confidence interval around the mean that is displayed.