Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Shadow block recovery mechanism #5296

Open
wants to merge 58 commits into
base: develop
Choose a base branch
from
Open

Feat: Shadow block recovery mechanism #5296

wants to merge 58 commits into from

Conversation

jcnelson
Copy link
Member

@jcnelson jcnelson commented Oct 10, 2024

This introduces the notion of "shadow blocks" and "shadow tenures" as a means of recovering from a chain stall. In the event of an irrecoverable chain stall, we'd pass an emergency SIP to declare that one or more empty sortitions actually commit to a "shadow block" that contains whatever state is needed to get the chain un-stuck. All nodes would ship with copies of these shadow blocks, and would process them the same as any other kind of block (except that they could not be mined, uploaded, or downloaded).

Leaving as a draft for now until I can expand test coverage. This touches a lot of consensus-critical code paths.


Here is a summary of consensus changes for shadow blocks:

  • All shadow blocks MUST have a version whose highest bit is set.
  • If any block in a tenure is a shadow block, then all blocks in that tenure MUST be shadow blocks (it's a "shadow tenure")
  • No non-shadow blocks can be inserted into a shadow tenure.
  • The block-commit of the first child tenure of a shadow block tenure MUST set parent_vtxindex to 0, which is the burnchain coinbase transaction. We assume that shadow blocks all descend from their respective PoX anchor blocks, so this will not be checked for the first child tenure. Instead, the block's parent sortition will be checked to verify that it is a shadow tenure and that the block-commit parent_vtxindex is 0 when validating the child block.
  • The VRF proof of a shadow block will not be validated (since there's no seed to use)
  • The miner signature of a shadow block will not be validated (since there's no miner)
  • For the purposes of finding the PoX anchor block, a shadow tenure in the prepare-phase always counts as a sortition, even if there were no block-commits in that burnchain block.
  • The signing weight of a shadow block is the maximal weight of the reward set. It's as if all signers signed it.
  • The coinbase of a shadow block MUST be the burn address for the network. Shadow blocks will produce STX, but no one will receive it.
  • The memo field of the coinbase of a shadow block MUST be the hash of the tenure-start block of the shadow block's parent tenure. This is required because shadow blocks do not have corresponding block-commits. Instead, we capture the tenure start block hash that would have gone into the block-commit as the memo field in the shadow block coinbase.
  • Shadow blocks do not increase the total BTC spend for the chain
  • The VRF proof for the child of a shadow block must come from the VRF seed of the shadow block -- i.e. the hash of the VRF proof in the parent shadow block's tenure's coinbase.

…there can be a test function called shadow_load_tenure_info() for shadow tenures
… via a SIP in order to fix a chain stall. Update the validation and processing logic to handle shadow blocks, since they won't have corresponding sortitions.
…reward (since there won't be any burns for it)
…t fetch them again. This would make it so we'd "discover" shadow tenures
…lly first and advance the tenure downloader state machine from that
…entory bitvec (required for the downloader to work)
Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

@jcnelson
Copy link
Member Author

N.B. This PR needs to be followed by (or extend with) a PR to re-mine a sequence of blocks as shadow blocks, so we can fix a consensus bug on testnet.

We'll need more elaborate chain replay logic in the future to preserve causal relationships between affected transactions if possible. At a minimum today, we can preserve the VRF proofs.

@kantai kantai changed the title Fix/5285 Feat: Shadow block recovery mechanism Oct 17, 2024
…t build atop shadow blocks point to the coinbase tx
… assumed to have descended from a shadow block, and thus the current PoX anchor block
…minnig after a shadow tenure (in which case ATC-C forbids us from winning)
…vailable if their shadow tenure isn't mined yet, and validate a block's block-commit against a parent shadow block's sortition (since we can't do that in LeaderBlockCommitOp::check_pox())
… that no blocks ever conflict with a shadow block
…s in ATC-C, and add functionality for mining shadow tenures
@jcnelson
Copy link
Member Author

Look at that.

All of that.

All those green ✔️'s in CI.

Outstanding work on making them all reliable @jferrant! It drastically improves our confidence that we can make consensus-critical changes like this. Well done!

@jcnelson jcnelson requested a review from obycode October 23, 2024 14:10
kantai
kantai previously approved these changes Oct 23, 2024
Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Just had some minor comments.

stackslib/src/net/tests/download/nakamoto.rs Outdated Show resolved Hide resolved
jferrant
jferrant previously approved these changes Oct 24, 2024
Co-authored-by: Brice Dobry <[email protected]>
@jcnelson jcnelson dismissed stale reviews from jferrant and kantai via dcab2b6 October 24, 2024 20:37
jferrant
jferrant previously approved these changes Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants