diff --git a/METHODOLOGY.md b/METHODOLOGY.md index 3603f69..2550be5 100644 --- a/METHODOLOGY.md +++ b/METHODOLOGY.md @@ -26,9 +26,13 @@ If the code and this document ever disagree, that is a bug — open an issue (se scoring both price freshness and manipulation resistance (§2), and the minimum-size filter §4 and §5 select reserves through ([The minimum-size filter](#the-minimum-size-filter)). -**Versioning begins here.** Every score in `risk_scores` carries -`methodology_version = 1`, because that is the only version any stored row has ever been -published under. There is no v1-versus-v2 boundary to look for, and no version-2 rows exist. +**Versioning begins here.** `methodology_version = 1` is the only version this rulebook +defines, and the only one any stored row will carry. A version 2 was briefly live in the code +— stamped onto runs between 2026-08-14 11:25 and 2026-08-18 11:30 UTC, before the rulebook was +flattened back to v1 — and that history is discarded rather than migrated, for the same reason +the development-era history was: it was computed under a rulebook that no longer exists, and +nobody was downstream of it. After that discard there is no v1-versus-v2 boundary in +`risk_scores`, and none to look for. ### Earlier development history was discarded, not migrated @@ -63,9 +67,9 @@ don't. ### Scores across a boundary are not comparable -There is no boundary in the stored data today, but the machinery that marks one is live and -tested, because the first bump must be legible on the day it happens rather than built in a -hurry then: +No boundary survives in the stored data — the v2 rows above were discarded — but the machinery +that marks one is live and tested, because the first real bump must be legible on the day it +happens rather than built in a hurry then: - The indexer stamps `risk_scores.methodology_version` from `METHODOLOGY_VERSION` in [`core/src/types.ts`](core/src/types.ts) at write time. An adapter has no say in it. @@ -157,19 +161,20 @@ even if no published number moved. #### Amendments folded into v1 Changes to the rulebook made **while v1 was still being finalized as the comparability -baseline.** These are not version boundaries: v1 is defined as the rulebook this document -describes, and these are part of that definition rather than a departure from it. But each one -moved a number that had already been stored, so each gets a row here. "No bump required" does -not mean "no record required" — a step in a history chart with nothing marking it is exactly -what the version stamp exists to prevent. +baseline** — before any surviving stored history existed. These are not version boundaries: v1 +is defined as the rulebook this document describes, and these are part of that definition +rather than a departure from it. None of them left a step in a published history, because the +history they predate was discarded rather than carried forward. Each gets a row anyway, so that +what v1 means is traceable rather than assumed: "no bump required" does not mean "no record +required". **This section closes when the next change lands.** From that point the rule in [What bumps the version](#what-bumps-the-version-going-forward) applies without exception, and a change that alters what a number means bumps to v2. -| Date (UTC) | Amendment | -| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `2026-08-18` | §4 and §5 gained the [minimum-size filter](#the-minimum-size-filter): both now select the worst reserve only among reserves clearing the protocol's own declared minimum exposure **or** 0.5% of the pool's supplied USD. This changes what the two factors measure, which is why it is recorded rather than treated as a correction. **No live score moved when it landed** — verified against both protocols on the day: Blend excludes nothing (its smallest reserve is ~$3.4M against a $5.00 `min_collateral`), and K2's dust reserve had already stopped being its worst reserve. **Stored K2 rows from before it are not comparable on these two factors:** on the frozen 2026-08-16 snapshot the filter moves `liquiditySafety` 34 → 44 and `utilizationSafety` 18 → 30 (score 24 → 28), by excluding a $3.00 reserve holding 0.19% of a $1,571 pool. | +| Date (UTC) | Amendment | +| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `2026-08-18` | §4 and §5 gained the [minimum-size filter](#the-minimum-size-filter): both now select the worst reserve only among reserves clearing the protocol's own declared minimum exposure **or** 0.5% of the pool's supplied USD. This changes what the two factors measure, which is why it is recorded rather than treated as a correction. **No live score moved when it landed** — verified against both protocols on the day: Blend excludes nothing (its smallest reserve is ~$3.4M against a $5.00 `min_collateral`), and K2's dust reserve had already stopped being its worst reserve. **What it changes, measured on the frozen 2026-08-16 snapshot:** `liquiditySafety` 34 → 44 and `utilizationSafety` 18 → 30 (score 24 → 28), by excluding a $3.00 reserve holding 0.19% of a $1,571 pool. | **History is not backfilled across a version bump, and cannot be.** `risk_scores` stores only outputs — the score and the factor map — never the raw on-chain inputs a run was @@ -743,6 +748,37 @@ closest to its line. --- +## Findings are published, not scored — and how they must be written + +Verifiable observations we can't or won't grade go in the protocol page's Findings section +(`dashboard/app/lib/protocol-notes.ts`), never into a factor. **Nothing there is read by any +scoring path**, and a note — favourable or not — can never move a number. + +**A note must survive the history it was drawn from.** Twice now a Findings note has outlived +the stored runs behind it: once when the development-era history was discarded, and again when +the briefly-live v2 rows were. Score history is not an archive — it is discarded across a +rulebook change and cannot be recomputed, because `risk_scores` keeps only outputs. A note +written as "our history shows X" therefore decays into an unverifiable claim on a page whose +entire pitch is that you don't have to trust us. + +So every note citing our own observations follows the same form: + +1. **Cite a closed window, with both ends stated.** "Between 2026-08-11 18:16 and 2026-08-18 + 15:55 UTC, 1,469 runs" — not "93% of runs", which silently means something different every + time the cron fires. A reader re-running the query later must be able to tell that a + different number is a later window, not a contradiction. +2. **Say the counts are a snapshot of that window** and do not update. +3. **Phrase the underlying claim so it stays checkable from chain after the history is gone.** + Our runs are evidence that a condition _persisted_; the condition itself must be one anyone + can observe today, directly from the contracts. If the only support for a claim is rows in + our database, it is not a finding — it is an assertion. +4. **Give the exact verification steps** — contract, method, field, and what to compare against. + If we can't say how a reader would check it themselves, it doesn't go in. +5. **Claim only what was measured.** Where a sub-signal wasn't recorded separately, say so and + scope the claim to the runs that carry it, rather than generalising across all of them. + +--- + ## Disputing or changing a threshold Every number in this document is meant to be challengeable — especially the ones labeled diff --git a/core/src/types.ts b/core/src/types.ts index d7141d8..c37cc17 100644 --- a/core/src/types.ts +++ b/core/src/types.ts @@ -16,13 +16,15 @@ export type Chain = 'stellar'; * 1 — the current five-factor rulebook, including `oracleSafety` scoring price * age *and* manipulation resistance. This is the first version anyone can * be downstream of: the development-era history that ran under earlier, - * unpublished iterations was discarded rather than migrated, so no stored - * row carries anything but 1. See METHODOLOGY.md, "Current version". + * unpublished iterations was discarded rather than migrated, and so is the + * history stamped 2 while a briefly-live v2 was in this constant before the + * rulebook was flattened back. See METHODOLOGY.md, "Current version". * - * There is no version 2 yet. The next change that alters what a number means - * makes one — and the machinery below (the stamp, the DB column, the chart's - * break rendering) exists and is tested precisely so that bump is legible on - * the day it happens, not built in a hurry then. + * The next change that alters what a number means makes version 2 — properly + * this time, as a published boundary rather than a value that came and went. + * The machinery below (the stamp, the DB column, the chart's break rendering) + * exists and is tested precisely so that bump is legible on the day it happens, + * not built in a hurry then. */ export const METHODOLOGY_VERSION = 1 as const; diff --git a/dashboard/app/globals.css b/dashboard/app/globals.css index 90d8f4d..61e7f7e 100644 --- a/dashboard/app/globals.css +++ b/dashboard/app/globals.css @@ -278,6 +278,27 @@ html { scroll-behavior: smooth; + + /* Reserve the scrollbar's gutter so a page that scrolls and a page that + doesn't lay out at the same width. + + Without this, every `mx-auto` container on the site — including the nav's + inner max-w-6xl — shifts sideways by half the scrollbar width whenever the + scrollbar appears or disappears. /methodology renders the whole of + METHODOLOGY.md and always scrolls; shorter routes may not, so navigating + between them visibly moves the navbar. + + It also covers a second shift: the mobile menu sets body.overflow = + 'hidden' while open (see nav.tsx), which propagates to the viewport and + removes the scrollbar. The gutter is reserved either way, so the page + underneath no longer jumps. + + Costs nothing where scrollbars overlay the content instead of taking + layout space (macOS, iOS, Android): `scrollbar-gutter` applies only to + classic scrollbars, so those platforms are unaffected. Where it does + apply, the strip is invisible — html has no background of its own, so the + canvas paints body's background across it. */ + scrollbar-gutter: stable; } body { diff --git a/dashboard/app/lib/protocol-notes.ts b/dashboard/app/lib/protocol-notes.ts index 43bf202..8279ef0 100644 --- a/dashboard/app/lib/protocol-notes.ts +++ b/dashboard/app/lib/protocol-notes.ts @@ -40,13 +40,17 @@ export const PROTOCOL_NOTES: Record = { 'It is not a continuous outage. The feed refreshes to an age of a few hundred ' + 'seconds, climbs back past the threshold over roughly an hour, and then sits ' + 'there for hours before refreshing again. That cycle is what makes the overall ' + - 'score oscillate, and it is visible as a repeating sawtooth in the score ' + - 'history above — the oscillation is the price ageing out and being renewed, ' + - 'not the pool’s risk genuinely changing every few minutes.', - 'These counts come from the live scored history the API serves, over the closed ' + - 'window stated above — they are a snapshot of that window and do not update as ' + - 'new runs land. The condition itself is checkable on-chain directly, by the ' + - 'steps below, without taking our history on trust.', + 'score oscillate: the oscillation is the price ageing out and being renewed, ' + + 'not the pool’s risk genuinely changing every few minutes. It shows up as a ' + + 'repeating sawtooth in the score history above once enough runs have ' + + 'accumulated since the history was reset — a chart covering only a few hours ' + + 'may catch one excursion, or none.', + 'The runs behind these counts have since been discarded — the stored score ' + + 'history was reset when the methodology was flattened to a single published ' + + 'version, so these figures cannot be re-derived from the API. What is stated ' + + 'here is an observation over a closed window that was made and recorded, not a ' + + 'claim about rows you can still fetch. The condition itself needs none of our ' + + 'history: it is checkable on-chain directly, by the steps below.', 'What is not being claimed: the circuit breaker was scored 100 throughout, so ' + 'the bound on a single-step price move is armed — this is purely about ' + 'freshness. And a price past a staleness threshold does not by itself mean the ' + diff --git a/db/src/store.test.ts b/db/src/store.test.ts index 07f94ac..a23abae 100644 --- a/db/src/store.test.ts +++ b/db/src/store.test.ts @@ -326,3 +326,92 @@ describe('toLeaderboardEntry', () => { assert.equal(entry.lastRunStatus, 'failed'); }); }); + +// --------------------------------------------------------------------------- +// The shape an EMPTY `risk_scores` table produces. +// +// WHY THIS EXISTS: both read queries LEFT JOIN LATERAL onto `risk_scores`, so +// wiping that table does not remove protocols from the API — it returns every +// protocol with every score-derived column null, including `last_run_at` and +// `last_run_status`, which the never-scored tests above still populate. That is +// a distinct row shape, it is what the site serves for the whole window between +// a history wipe and the next indexer run, and it is otherwise only reachable in +// production for a few minutes at a time. Pin it here rather than discover it +// there. +// --------------------------------------------------------------------------- + +describe('an empty risk_scores table', () => { + /** Exactly what the LATERAL joins yield when the table has no rows at all. */ + const wipedDetail = (): ProtocolDetailRow => + detailRow({ + safety_score: null, + computed_at: null, + factors: null, + methodology_version: null, + last_run_at: null, + last_run_status: null, + }); + + it('still returns the protocol, rather than dropping or 404ing it', () => { + // The API 404s on an unknown id. A known protocol with no scores is not + // unknown, and must not become so — the detail route reads `!detail`, which + // is only null when the `protocols` row itself is missing. + const detail = toProtocolDetail(wipedDetail(), []); + assert.equal(detail.id, 'blend'); + assert.equal(detail.name, 'Blend'); + assert.equal(detail.adapter, 'BlendAdapter'); + }); + + it('nulls every score-derived field and empties the history', () => { + const detail = toProtocolDetail(wipedDetail(), []); + assert.deepEqual( + { + safetyScore: detail.safetyScore, + computedAt: detail.computedAt, + factors: detail.factors, + methodologyVersion: detail.methodologyVersion, + lastRunAt: detail.lastRunAt, + lastRunStatus: detail.lastRunStatus, + history: detail.history, + }, + { + safetyScore: null, + computedAt: null, + factors: null, + methodologyVersion: null, + lastRunAt: null, + lastRunStatus: null, + history: [], + }, + ); + }); + + it('keeps the verification links a reader needs precisely when there is no score', () => { + // With no number to show, the contract link is the only thing on the page + // that still lets someone check the protocol themselves. + const detail = toProtocolDetail(wipedDetail(), []); + assert.equal(detail.contractId, 'CAJJZSGMMM3PD7N33TAPHGBUGTB43OC73HVIK2L2G6BNGGGYOSSYBXBD'); + assert.equal(detail.site, 'https://www.blend.capital'); + assert.equal(detail.logo, '/assets/protocols/blend.svg'); + }); + + it('produces a leaderboard entry the UI reads as "never run"', () => { + // `lastRunStatus: null` is what drives freshness() to the "never run" + // branch, and `safetyScore: null` is what makes ScoreRing render an em dash + // instead of a zero. A wiped table must not look like a pool that scored 0. + const entry = toLeaderboardEntry({ + id: 'blend', + name: 'Blend', + chain: 'stellar', + logo: '/assets/protocols/blend.svg', + safety_score: null, + computed_at: null, + last_run_at: null, + last_run_status: null, + }); + assert.equal(entry.safetyScore, null, 'null, never 0 — 0 means "scored, and unsafe"'); + assert.equal(entry.lastRunStatus, null); + assert.equal(entry.computedAt, null); + assert.equal(entry.name, 'Blend'); + }); +});