Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 50 additions & 14 deletions METHODOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,13 @@ If the code and this document ever disagree, that is a bug — open an issue (se
scoring both price freshness and manipulation resistance (§2), and the minimum-size filter §4
and §5 select reserves through ([The minimum-size filter](#the-minimum-size-filter)).

**Versioning begins here.** Every score in `risk_scores` carries
`methodology_version = 1`, because that is the only version any stored row has ever been
published under. There is no v1-versus-v2 boundary to look for, and no version-2 rows exist.
**Versioning begins here.** `methodology_version = 1` is the only version this rulebook
defines, and the only one any stored row will carry. A version 2 was briefly live in the code
— stamped onto runs between 2026-08-14 11:25 and 2026-08-18 11:30 UTC, before the rulebook was
flattened back to v1 — and that history is discarded rather than migrated, for the same reason
the development-era history was: it was computed under a rulebook that no longer exists, and
nobody was downstream of it. After that discard there is no v1-versus-v2 boundary in
`risk_scores`, and none to look for.

### Earlier development history was discarded, not migrated

Expand Down Expand Up @@ -63,9 +67,9 @@ don't.

### Scores across a boundary are not comparable

There is no boundary in the stored data today, but the machinery that marks one is live and
tested, because the first bump must be legible on the day it happens rather than built in a
hurry then:
No boundary survives in the stored data the v2 rows above were discarded — but the machinery
that marks one is live and tested, because the first real bump must be legible on the day it
happens rather than built in a hurry then:

- The indexer stamps `risk_scores.methodology_version` from `METHODOLOGY_VERSION` in
[`core/src/types.ts`](core/src/types.ts) at write time. An adapter has no say in it.
Expand Down Expand Up @@ -157,19 +161,20 @@ even if no published number moved.
#### Amendments folded into v1

Changes to the rulebook made **while v1 was still being finalized as the comparability
baseline.** These are not version boundaries: v1 is defined as the rulebook this document
describes, and these are part of that definition rather than a departure from it. But each one
moved a number that had already been stored, so each gets a row here. "No bump required" does
not mean "no record required" — a step in a history chart with nothing marking it is exactly
what the version stamp exists to prevent.
baseline** — before any surviving stored history existed. These are not version boundaries: v1
is defined as the rulebook this document describes, and these are part of that definition
rather than a departure from it. None of them left a step in a published history, because the
history they predate was discarded rather than carried forward. Each gets a row anyway, so that
what v1 means is traceable rather than assumed: "no bump required" does not mean "no record
required".

**This section closes when the next change lands.** From that point the rule in
[What bumps the version](#what-bumps-the-version-going-forward) applies without exception, and
a change that alters what a number means bumps to v2.

| Date (UTC) | Amendment |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `2026-08-18` | §4 and §5 gained the [minimum-size filter](#the-minimum-size-filter): both now select the worst reserve only among reserves clearing the protocol's own declared minimum exposure **or** 0.5% of the pool's supplied USD. This changes what the two factors measure, which is why it is recorded rather than treated as a correction. **No live score moved when it landed** — verified against both protocols on the day: Blend excludes nothing (its smallest reserve is ~$3.4M against a $5.00 `min_collateral`), and K2's dust reserve had already stopped being its worst reserve. **Stored K2 rows from before it are not comparable on these two factors:** on the frozen 2026-08-16 snapshot the filter moves `liquiditySafety` 34 → 44 and `utilizationSafety` 18 → 30 (score 24 → 28), by excluding a $3.00 reserve holding 0.19% of a $1,571 pool. |
| Date (UTC) | Amendment |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `2026-08-18` | §4 and §5 gained the [minimum-size filter](#the-minimum-size-filter): both now select the worst reserve only among reserves clearing the protocol's own declared minimum exposure **or** 0.5% of the pool's supplied USD. This changes what the two factors measure, which is why it is recorded rather than treated as a correction. **No live score moved when it landed** — verified against both protocols on the day: Blend excludes nothing (its smallest reserve is ~$3.4M against a $5.00 `min_collateral`), and K2's dust reserve had already stopped being its worst reserve. **What it changes, measured on the frozen 2026-08-16 snapshot:** `liquiditySafety` 34 → 44 and `utilizationSafety` 18 → 30 (score 24 → 28), by excluding a $3.00 reserve holding 0.19% of a $1,571 pool. |

**History is not backfilled across a version bump, and cannot be.** `risk_scores` stores
only outputs — the score and the factor map — never the raw on-chain inputs a run was
Expand Down Expand Up @@ -743,6 +748,37 @@ closest to its line.

---

## Findings are published, not scored — and how they must be written

Verifiable observations we can't or won't grade go in the protocol page's Findings section
(`dashboard/app/lib/protocol-notes.ts`), never into a factor. **Nothing there is read by any
scoring path**, and a note — favourable or not — can never move a number.

**A note must survive the history it was drawn from.** Twice now a Findings note has outlived
the stored runs behind it: once when the development-era history was discarded, and again when
the briefly-live v2 rows were. Score history is not an archive — it is discarded across a
rulebook change and cannot be recomputed, because `risk_scores` keeps only outputs. A note
written as "our history shows X" therefore decays into an unverifiable claim on a page whose
entire pitch is that you don't have to trust us.

So every note citing our own observations follows the same form:

1. **Cite a closed window, with both ends stated.** "Between 2026-08-11 18:16 and 2026-08-18
15:55 UTC, 1,469 runs" — not "93% of runs", which silently means something different every
time the cron fires. A reader re-running the query later must be able to tell that a
different number is a later window, not a contradiction.
2. **Say the counts are a snapshot of that window** and do not update.
3. **Phrase the underlying claim so it stays checkable from chain after the history is gone.**
Our runs are evidence that a condition _persisted_; the condition itself must be one anyone
can observe today, directly from the contracts. If the only support for a claim is rows in
our database, it is not a finding — it is an assertion.
4. **Give the exact verification steps** — contract, method, field, and what to compare against.
If we can't say how a reader would check it themselves, it doesn't go in.
5. **Claim only what was measured.** Where a sub-signal wasn't recorded separately, say so and
scope the claim to the runs that carry it, rather than generalising across all of them.

---

## Disputing or changing a threshold

Every number in this document is meant to be challengeable — especially the ones labeled
Expand Down
14 changes: 8 additions & 6 deletions core/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,15 @@ export type Chain = 'stellar';
* 1 — the current five-factor rulebook, including `oracleSafety` scoring price
* age *and* manipulation resistance. This is the first version anyone can
* be downstream of: the development-era history that ran under earlier,
* unpublished iterations was discarded rather than migrated, so no stored
* row carries anything but 1. See METHODOLOGY.md, "Current version".
* unpublished iterations was discarded rather than migrated, and so is the
* history stamped 2 while a briefly-live v2 was in this constant before the
* rulebook was flattened back. See METHODOLOGY.md, "Current version".
*
* There is no version 2 yet. The next change that alters what a number means
* makes one — and the machinery below (the stamp, the DB column, the chart's
* break rendering) exists and is tested precisely so that bump is legible on
* the day it happens, not built in a hurry then.
* The next change that alters what a number means makes version 2 — properly
* this time, as a published boundary rather than a value that came and went.
* The machinery below (the stamp, the DB column, the chart's break rendering)
* exists and is tested precisely so that bump is legible on the day it happens,
* not built in a hurry then.
*/
export const METHODOLOGY_VERSION = 1 as const;

Expand Down
21 changes: 21 additions & 0 deletions dashboard/app/globals.css
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,27 @@

html {
scroll-behavior: smooth;

/* Reserve the scrollbar's gutter so a page that scrolls and a page that
doesn't lay out at the same width.

Without this, every `mx-auto` container on the site — including the nav's
inner max-w-6xl — shifts sideways by half the scrollbar width whenever the
scrollbar appears or disappears. /methodology renders the whole of
METHODOLOGY.md and always scrolls; shorter routes may not, so navigating
between them visibly moves the navbar.

It also covers a second shift: the mobile menu sets body.overflow =
'hidden' while open (see nav.tsx), which propagates to the viewport and
removes the scrollbar. The gutter is reserved either way, so the page
underneath no longer jumps.

Costs nothing where scrollbars overlay the content instead of taking
layout space (macOS, iOS, Android): `scrollbar-gutter` applies only to
classic scrollbars, so those platforms are unaffected. Where it does
apply, the strip is invisible — html has no background of its own, so the
canvas paints body's background across it. */
scrollbar-gutter: stable;
}

body {
Expand Down
18 changes: 11 additions & 7 deletions dashboard/app/lib/protocol-notes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,17 @@ export const PROTOCOL_NOTES: Record<string, ProtocolNote[]> = {
'It is not a continuous outage. The feed refreshes to an age of a few hundred ' +
'seconds, climbs back past the threshold over roughly an hour, and then sits ' +
'there for hours before refreshing again. That cycle is what makes the overall ' +
'score oscillate, and it is visible as a repeating sawtooth in the score ' +
'history above — the oscillation is the price ageing out and being renewed, ' +
'not the pool’s risk genuinely changing every few minutes.',
'These counts come from the live scored history the API serves, over the closed ' +
'window stated above — they are a snapshot of that window and do not update as ' +
'new runs land. The condition itself is checkable on-chain directly, by the ' +
'steps below, without taking our history on trust.',
'score oscillate: the oscillation is the price ageing out and being renewed, ' +
'not the pool’s risk genuinely changing every few minutes. It shows up as a ' +
'repeating sawtooth in the score history above once enough runs have ' +
'accumulated since the history was reset — a chart covering only a few hours ' +
'may catch one excursion, or none.',
'The runs behind these counts have since been discarded — the stored score ' +
'history was reset when the methodology was flattened to a single published ' +
'version, so these figures cannot be re-derived from the API. What is stated ' +
'here is an observation over a closed window that was made and recorded, not a ' +
'claim about rows you can still fetch. The condition itself needs none of our ' +
'history: it is checkable on-chain directly, by the steps below.',
'What is not being claimed: the circuit breaker was scored 100 throughout, so ' +
'the bound on a single-step price move is armed — this is purely about ' +
'freshness. And a price past a staleness threshold does not by itself mean the ' +
Expand Down
89 changes: 89 additions & 0 deletions db/src/store.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,3 +326,92 @@ describe('toLeaderboardEntry', () => {
assert.equal(entry.lastRunStatus, 'failed');
});
});

// ---------------------------------------------------------------------------
// The shape an EMPTY `risk_scores` table produces.
//
// WHY THIS EXISTS: both read queries LEFT JOIN LATERAL onto `risk_scores`, so
// wiping that table does not remove protocols from the API — it returns every
// protocol with every score-derived column null, including `last_run_at` and
// `last_run_status`, which the never-scored tests above still populate. That is
// a distinct row shape, it is what the site serves for the whole window between
// a history wipe and the next indexer run, and it is otherwise only reachable in
// production for a few minutes at a time. Pin it here rather than discover it
// there.
// ---------------------------------------------------------------------------

describe('an empty risk_scores table', () => {
/** Exactly what the LATERAL joins yield when the table has no rows at all. */
const wipedDetail = (): ProtocolDetailRow =>
detailRow({
safety_score: null,
computed_at: null,
factors: null,
methodology_version: null,
last_run_at: null,
last_run_status: null,
});

it('still returns the protocol, rather than dropping or 404ing it', () => {
// The API 404s on an unknown id. A known protocol with no scores is not
// unknown, and must not become so — the detail route reads `!detail`, which
// is only null when the `protocols` row itself is missing.
const detail = toProtocolDetail(wipedDetail(), []);
assert.equal(detail.id, 'blend');
assert.equal(detail.name, 'Blend');
assert.equal(detail.adapter, 'BlendAdapter');
});

it('nulls every score-derived field and empties the history', () => {
const detail = toProtocolDetail(wipedDetail(), []);
assert.deepEqual(
{
safetyScore: detail.safetyScore,
computedAt: detail.computedAt,
factors: detail.factors,
methodologyVersion: detail.methodologyVersion,
lastRunAt: detail.lastRunAt,
lastRunStatus: detail.lastRunStatus,
history: detail.history,
},
{
safetyScore: null,
computedAt: null,
factors: null,
methodologyVersion: null,
lastRunAt: null,
lastRunStatus: null,
history: [],
},
);
});

it('keeps the verification links a reader needs precisely when there is no score', () => {
// With no number to show, the contract link is the only thing on the page
// that still lets someone check the protocol themselves.
const detail = toProtocolDetail(wipedDetail(), []);
assert.equal(detail.contractId, 'CAJJZSGMMM3PD7N33TAPHGBUGTB43OC73HVIK2L2G6BNGGGYOSSYBXBD');
assert.equal(detail.site, 'https://www.blend.capital');
assert.equal(detail.logo, '/assets/protocols/blend.svg');
});

it('produces a leaderboard entry the UI reads as "never run"', () => {
// `lastRunStatus: null` is what drives freshness() to the "never run"
// branch, and `safetyScore: null` is what makes ScoreRing render an em dash
// instead of a zero. A wiped table must not look like a pool that scored 0.
const entry = toLeaderboardEntry({
id: 'blend',
name: 'Blend',
chain: 'stellar',
logo: '/assets/protocols/blend.svg',
safety_score: null,
computed_at: null,
last_run_at: null,
last_run_status: null,
});
assert.equal(entry.safetyScore, null, 'null, never 0 — 0 means "scored, and unsafe"');
assert.equal(entry.lastRunStatus, null);
assert.equal(entry.computedAt, null);
assert.equal(entry.name, 'Blend');
});
});
Loading