Skip to content

Commit c7d5cd0

Browse files
cmdcolinclaude
andcommitted
Trim source comments to what the code doesn't say
Comments made up about half of src, mostly history and measurements that docs/ already records. Keep one line on each non-obvious choice and point to docs/dataflow.md and docs/memory.md for the rest. docs/memory.md gains the one reason it lacked: the recency counter lives on SharedBudget because loading both the ESM and CJS builds would create two module-level counters. Two numbers that appeared only in comments, kept here for the record: - settledCount is maintained rather than counted because evict() needs it on every settle; counting cost 8.6us per read over 100 entries and 134us over 20,000, against 7.4us and 14us maintained. - SharedBudget.total prunes on read because an unlimited budget once never pruned and reported 10,200 against members holding 400. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 85f4dda commit c7d5cd0

4 files changed

Lines changed: 96 additions & 454 deletions

File tree

‎docs/memory.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,12 @@ touch, giving a total order across _different_ caches, which a shared budget
158158
compares. `lastTouched` answers "how long since" and `seq` answers "which came
159159
first", so the cache keeps both.
160160

161+
`SharedBudget` holds the counter, because a module-level counter can exist
162+
twice. A consumer whose dependency graph loads both the ESM and the CJS build
163+
gets two copies of the module, each counting from zero, and a cache from each
164+
copy then stamps entries in one budget with colliding numbers. In that setup a
165+
module counter evicted the second-newest entry of seven and kept the oldest.
166+
161167
For the same reason, a read settling stamps `lastTouched` but deliberately does
162168
not `touch()` the entry. A read's latency is a property of the transport, not of
163169
how the consumer is using the cache, so ordering evictions by it preferentially

‎src/SharedBudget.ts‎

Lines changed: 24 additions & 148 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,10 @@
1-
/**
2-
* What a {@link SharedBudget} needs from a cache. Implemented by
3-
* {@link SharedReadCache}; there is no reason for a consumer to write one.
4-
*/
1+
/** What a {@link SharedBudget} needs from a cache. */
52
export interface BudgetMember {
63
/**
7-
* The least-recently-used settled entry this member is willing to give up,
8-
* or `undefined` if it has none to spare. `last` is true when that entry is
9-
* the member's only settled one.
10-
*
11-
* Without `offerLast` a member keeps its last settled entry. A budget passes
12-
* it, because one kept entry per member lets N members hold N entries
13-
* whatever the limit; the budget keeps one across all its members instead.
14-
* Both the parameter and `last` are optional, so a member and a budget from
15-
* different versions of this package still pair up.
4+
* The least-recently-used settled entry this member will give up, with
5+
* `last` set when it is the member's only one. Without `offerLast` a member
6+
* keeps its last entry. Both are optional so that members and budgets from
7+
* different installs of this package still pair up.
168
*/
179
lruSpare(
1810
offerLast?: boolean,
@@ -21,157 +13,58 @@ export interface BudgetMember {
2113
release(cacheKey: string): void
2214
}
2315

24-
/**
25-
* The handle a member reports its weight through. Opaque to the member: it
26-
* passes back to {@link Budget.charge} the one {@link Budget.register} gave it.
27-
*/
16+
/** The handle a member reports its weight through. */
2817
export interface BudgetMembership {
2918
/** weight this member currently contributes to the budget's total */
3019
held: number
3120
}
3221

3322
/**
34-
* What a {@link SharedReadCache} needs from the budget it joins, and the type
35-
* of its `budget` option.
36-
*
37-
* An interface rather than the {@link SharedBudget} class so that the budget
38-
* and the cache need not come from one copy of this package. An application
39-
* creates the budget from its own install and hands it to caches constructed
40-
* inside `@gmod/bam` or `@gmod/tabix` from theirs; a class with private fields
41-
* is nominal, so two installs of even the same version would not typecheck
42-
* against each other. Changing this interface breaks that pairing at runtime,
43-
* so it is semver-major surface.
23+
* The type of a cache's `budget` option. An interface rather than the
24+
* {@link SharedBudget} class, so the budget and its caches can come from
25+
* different installs of this package; changing it is a breaking change.
4426
*/
4527
export interface Budget {
4628
register(member: BudgetMember): BudgetMembership
47-
/** Adjust a member's contribution, and the total with it. */
4829
charge(membership: BudgetMembership, delta: number): void
4930
/** The next stamp in the recency order shared by every member. */
5031
nextSeq(): number
51-
/** Evict globally least-recently-used entries until back under the limit. */
5232
evict(): void
5333
}
5434

55-
/**
56-
* The budget keeps this so that a cache which has been collected can still have
57-
* its last known weight subtracted from the total.
58-
*/
5935
interface Membership extends BudgetMembership {
6036
ref: WeakRef<BudgetMember>
6137
}
6238

6339
/**
6440
* One memory budget shared by several caches, evicting globally
65-
* least-recently-used across all of them.
66-
*
67-
* ## Why this exists
68-
*
69-
* {@link SharedReadCacheOptions.maxSize} is per cache, and a consumer that
70-
* opens one cache per file multiplies it. @gmod/bam's 1 GB default is sized so
71-
* a single track's pan never thrashes; jbrowse holds one `BamFile` per open
72-
* track, so three moderately deep alignments tracks browsing eight windows
73-
* measured 1109 MB retained and 1665 MB RSS — with **no cache anywhere near
74-
* its own ceiling**, so not one byte of that was the budget's doing. Per-file
75-
* ceilings do not bound a consumer that scales the number of files.
76-
*
77-
* ## Why not just divide the ceiling by the file count
78-
*
79-
* Because that walks into the cliff the per-file number exists to avoid. A
80-
* budget below one query's working set does not cache less, it caches
81-
* *nothing* — each value is evicted before the next pan can reuse it, so the
82-
* hit rate is zero and the memory is retained anyway. On that same three-track
83-
* workload, 1 GB split as 342 MB each cost 16 refills on the revisit; split
84-
* eight ways as 128 MB each it cost 101, against 98 for the cold pass. Worse
85-
* than no cache at all.
86-
*
87-
* A shared budget does not have that failure, because a member yields only
88-
* what is globally least-recently-used. Tracks the reader is not looking at
89-
* age out and hand their space to the one being panned, so the active track
90-
* keeps a working set whatever the track count — which is exactly what an
91-
* equal split cannot do.
92-
*
93-
* ## Every member must weigh in the same unit
94-
*
95-
* {@link total} is a sum over members, so it means nothing unless their
96-
* {@link SharedReadCacheOptions.sizeOf} agree. This is not hypothetical across
97-
* these packages: @gmod/bam, @gmod/tabix and @gmod/cram weigh decompressed
98-
* bytes, while @gmod/bbi, @gmod/nclist and @gmod/hic weigh entries. A budget
99-
* holding a bam cache and a bbi cache would be adding bytes to entries and
100-
* bounding neither.
101-
*
102-
* Nothing here can check that, since `sizeOf` is opaque by design. Group
103-
* members by unit and give each group its own budget.
104-
*
105-
* ## Members are held weakly
106-
*
107-
* A long-lived budget must never be the reason a cache stays reachable. The
108-
* consumer this is for keeps one budget per worker and one cache per open
109-
* track, and closing a track reclaims by dropping the last strong reference to
110-
* the adapter — a budget holding its members strongly would silently convert
111-
* that into a leak, which is the exact bug it is meant to prevent. So the
112-
* budget holds a `WeakRef` to each cache and its last known weight beside it;
113-
* a collected member is pruned, and its weight credited back, on the next
114-
* eviction pass. No `unregister` call for a consumer to forget.
41+
* least-recently-used across all of them. Every member must weigh in the same
42+
* unit. Members are held weakly, and a collected member's weight is credited
43+
* back on the next pass. See docs/memory.md.
11544
*/
11645
export class SharedBudget implements Budget {
11746
private members = new Set<Membership>()
11847
private budgetLimit: number
119-
/**
120-
* Ticks on every touch across every member, giving the total order over
121-
* entries that {@link evict} picks its victim by.
122-
*
123-
* `lastTouched` cannot do that job. `Date.now()` has millisecond resolution,
124-
* so a burst of cache hits inside one millisecond all carry the same stamp,
125-
* and a tie resolves to whichever member happened to be scanned last rather
126-
* than to anything about recency. Wall-clock is still what a cache's idle
127-
* sweep needs, so the two coexist: one answers "how long since", the other
128-
* "which came first".
129-
*
130-
* It lives here, on the object the members share, rather than in a module
131-
* variable beside the cache. This package ships an ESM build and a CJS one,
132-
* so a consumer whose dependency graph reaches both gets two copies of that
133-
* module and two counters, each starting at zero — and members from the two
134-
* then offered this budget colliding stamps. Measured on one budget holding
135-
* a cache from each build: it evicted the second-newest entry of seven and
136-
* kept the genuinely oldest, which is the below-the-working-set cliff this
137-
* class exists to avoid, arrived at through a build artifact rather than
138-
* through any configuration. A shared budget is by definition one object, so
139-
* a counter on it is one counter however many copies of the class exist.
140-
*/
48+
// a module-level counter would exist twice when a consumer loads both the
49+
// ESM and CJS builds; see docs/memory.md
14150
private seq = 0
142-
14351
private heldTotal = 0
14452

145-
/**
146-
* Sum of the settled weight held across every member.
147-
*
148-
* An accessor, because a member that has been collected owes nothing and this
149-
* is the number a consumer reports memory from. As a plain field it was only
150-
* corrected when something happened to call {@link prune} — {@link evict} or
151-
* {@link size} — so a budget sitting under its limit, which is a budget with
152-
* no reason to evict, went on counting caches that no longer exist for as
153-
* long as nothing else happened. That is the same 10,200-against-400 the
154-
* pruning order inside {@link evict} was fixed for, reached by sitting still
155-
* rather than by having no limit.
156-
*/
53+
constructor(limit: number) {
54+
this.budgetLimit = limit
55+
}
56+
57+
/** Settled weight held across every live member. */
15758
get total() {
15859
this.prune()
15960
return this.heldTotal
16061
}
16162

162-
constructor(limit: number) {
163-
this.budgetLimit = limit
164-
}
165-
16663
get limit() {
16764
return this.budgetLimit
16865
}
16966

170-
/**
171-
* Accessor rather than a plain field for the same reason
172-
* {@link SharedReadCache.maxSize} is one: lowering the budget frees memory
173-
* now, rather than whenever some member next happens to settle a read.
174-
*/
67+
/** Lowering the limit evicts now. */
17568
set limit(limit: number) {
17669
this.budgetLimit = limit
17770
this.evict()
@@ -183,43 +76,29 @@ export class SharedBudget implements Budget {
18376
return this.members.size
18477
}
18578

186-
/**
187-
* Enrol a cache. Returns the handle it reports its weight through — the
188-
* budget never asks a member for its weight, because a member that has been
189-
* collected still owes what it held.
190-
*/
19179
register(member: BudgetMember): BudgetMembership {
19280
const membership = { held: 0, ref: new WeakRef(member) }
19381
this.members.add(membership)
19482
return membership
19583
}
19684

197-
/**
198-
* @internal — the next stamp in this budget's recency order. Members ask on
199-
* every touch; nothing else should.
200-
*/
85+
/** @internal */
20186
nextSeq() {
20287
return this.seq++
20388
}
20489

205-
/** Adjust a member's contribution, and the total with it. */
20690
charge(membership: BudgetMembership, delta: number) {
20791
membership.held += delta
20892
this.heldTotal += delta
20993
}
21094

21195
/**
21296
* Evict globally least-recently-used settled entries until back under the
213-
* limit, or until no member will give up another.
97+
* limit, keeping reads in flight and the one newest settled entry.
21498
*/
21599
evict() {
216-
// Before the limit is consulted, because pruning is not part of evicting:
217-
// {@link total} is a documented public number and a member that has been
218-
// collected owes nothing, whether or not there is a limit to compare it
219-
// against. Guarded the other way round, an unlimited budget returned here
220-
// and so never pruned at all — it reported 10,200 against members actually
221-
// holding 400, and kept a Membership for every cache ever registered, which
222-
// is the accumulation the weak refs exist to avoid.
100+
// before the limit check, so an unlimited budget still drops collected
101+
// members
223102
this.prune()
224103
while (this.heldTotal > this.budgetLimit) {
225104
let victim: BudgetMember | undefined
@@ -236,16 +115,13 @@ export class SharedBudget implements Budget {
236115
}
237116
}
238117
}
239-
// Everything left is a read in flight, or the one settled entry the
240-
// budget keeps across all its members for the request in flight.
241118
if (!victim || !spare || (offers === 1 && spare.last)) {
242119
break
243120
}
244121
victim.release(spare.cacheKey)
245122
}
246123
}
247124

248-
/** Drop collected members, crediting back what they were still counted for. */
249125
private prune() {
250126
for (const membership of this.members) {
251127
if (membership.ref.deref() === undefined) {

0 commit comments

Comments
 (0)