You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5e70e39
Browse filesBrowse the repository at this point in the historyBrowse files
The owner raised the per-case time limit from 6 to 15 minutes. It still applies
the same way to every database, and loads still always finish.
Verified: Debug build of KeyLoad.ComparisonHost with warnings as errors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/Features/BenchmarkComparisons.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ decision is [ADR-150](../ADR/ADR-150-per-type-benchmark-jobs.md).
11
11
|---|---|
12
12
| REQ-BC-030: one job per database, data type and node count |`benchmarks.yml` has one `measure` job per row of the [type table](#data-types); a job deploys one database and runs only the TUnit class of its type. |
13
13
| REQ-BC-031: one shared module per data type | The dataset, scenario (cases, schedule, verification) and driver interface of a type exist once; a database implements only the driver. |
14
-
| REQ-BC-032: the same standard size everywhere | Seed 1729, 16 concurrent clients, 100,000 planned operations per measured case, at most 6 minutes per measured case, and 1,000,000-record datasets, unless a local run overrides them. Dataset loads are never stopped by the limit. |
14
+
| REQ-BC-032: the same standard size everywhere | Seed 1729, 16 concurrent clients, 100,000 planned operations per measured case, at most 15 minutes per measured case, and 1,000,000-record datasets, unless a local run overrides them. Dataset loads are never stopped by the limit. |
15
15
| REQ-BC-033: results survive failure | The job writes its result file after startup, after every case and on disposal, and uploads it whether the job passed or failed. |
16
16
| REQ-BC-034: no invented numbers | A failed case keeps its real first error; a case that never ran, a missing node count or an unsupported database has no number. |
17
17
| REQ-BC-035: the website shows one run | Website publishes the result files of the dispatched run, or of the latest completed run; without result files it publishes nothing and the current Pages stay. |
@@ -136,9 +136,8 @@ latency per call over successful calls; 0 when no call succeeded), `batchSize`
136
136
fails when any unit fails or its after-the-fact verification finds a problem, and
137
137
the TUnit test fails with it. No value is ever estimated or filled in.
138
138
139
-
The case time limit is the mean measured case time across every database of the
140
-
first per-type run (5.8 minutes, rounded to 6). It applies the same way to every
141
-
database: once it has passed, clients take no new operation, operations already
139
+
The case time limit is 15 minutes, set by the owner. It applies the same way to
140
+
every database: once it has passed, clients take no new operation, operations already
142
141
started finish, and `opsPerSecond` and the latencies cover only what ran. The
143
142
site shows such a case as "<ran> of <planned>". Loads always finish, because the
144
143
following cases read the loaded data. `Benchmarks__CaseMinutes` overrides the
0 commit comments