Ledger state: governance

[KtorZ]

What

We need to augment Amaru's ledger state to support Governance. This is a rather large chunk and roughly encompasses:

• Managing DReps registrations & delegation, similar to what we did for Stake Pools;
• Computing DRep stake distribution (incrementally or at the epoch boundary);
• Tracking governance actions;
• Counting votes during the epoch;
• Ratifying and enacting governance action.

One particularly useful element at first is the tracking of deposits (DReps & proposals) as it directly impact the stake distribution (and thus the other stream of work we are currently pursuing w.r.t the leader schedule validation). While deposits are relatively easy to track, the refund demands a comprehension of the ratification process for proposals.

Caution

We currently limit the scope to merely tracking the state. We do not yet seek to implement all ledger rules and "input validations", only what's necessary to keep the state up-to-date. We'll eventually have to implement the full ledger rules, but this is consider out-of-scope of this particular task.

Functional Requirements

Block processing time <1s

As for other operations within Amaru's ledger, we must ensure their prompt execution. While in this initial phase, we are purposely not worrying too much about the time needed for operations at an epoch boundary, we still strive for avoiding doing too much work at an epoch boundary.

While we can relatively easily dodge the bullet for stake pool stake distribution since the result of the reward calculation is only needed in the next epoch, we need the DRep stake distribution at the end of the current epoch boundary. Doing this calculation right at the end may likely be too expensive; so it's likely something we need done incrementally (eventually).

On-disk persistence

As for other elements of the ledger state, the governance state shall be tracked and stored on-disk through a mix of the current RocksDB backend for the stable parts, and some in-memory representation regarding the volatile part (most recent k = 2160 blocks).

Given the constraints on block processing at the epoch boundary, it is probably a good idea to keep the persisted format somewhat optimized towards querying and processing for the end of the epoch (in opposition to keeping it optimized for writing block after block). Though as always, measuring and monitoring will tell us what to do and improve.

Useful links

• Cardano's ledger Agda specification
• Transition rule implementation for:
  • Governance
  • Certificates
  • Epoch transition
  • Ratification
• Haskell's ledger internal data-structures & operators related to governance
  • Unified Map (a.k.a UMap)
  • GovRelation
  • GovActionState
  • DRep Pulser (incremental calculation of DRep stake distr)
  • Definitions of operators for set algebra (often used across the ledger codebase)
• (possibly useful) Local state queries related to governance

[KtorZ]

cc @jeluard let's go through that together, and feel free to drop questions here.

[Quantumplation]

Investigating when/how gov proposals get refunded;

Looks like the entrypoint is in the epoch transition. I'm not great at reading Haskell, but the order of events seems to be:

1. Snapshot the governance state, protocol parameters, utxo state, and certificate state
2. Activate future stake pools
3. Get stake pool and dRep stake distributions
4. Process any treasury withdrawals
5. Update the tree of proposals, removing conflicts from enacted proposals and removing expired proposals
   a. This is implemented here
6. Apply the changes from the enacted proposals to the governance state
7. Calculate the proposal deposits that need to be returned
   a. This is implemented here
8. Emit a ledger event for enacted actions, expired actions, and unclaimed deposits
9. Update the reward accounts with the returned deposits, I think?? this is the only place newUMap is updated
10. Increment the epoch count for ongoing proposals
11. Remove CC members
12. Update the treasury with donations and unclaimed gov deposits
13. A bunch of stuff I don't really understand, likely updating relevant state updates?
14. Update protocol parameters and initiate a hard fork

Most notably, step 7 and step 9 are where the deposits are computed and applied to the new state.

In step 7, this is on all removedGovernanceActions which is computed in step 5, in proposalsApplyEnactment, implemented here.

Gunna pause here, but next I'll dig into the criteria that determines if a proposal is enacted, removed, and expired.

[jeluard]

That is very helpful @Quantumplation , thanks!

[Quantumplation]

np! I tried reading through the haskell code to see what criteria determines an expired action and it's a bit beyond my haskell skills. I know it's some number of epochs from the protocol parameters, but i can't read the code to determine the exact behavior, so I'd probably commit an off-by-one error in trying to describe it.

I'm just kind of casually picking away at chores in my spare time, so if you have anything you want to delegate to me feel free to throw it my way :) Happy to write annoying / repetitive test cases etc.