[feat] adds simulation to block loop #71

dylanlott · 2025-04-15T19:36:49Z

[feat] adds simulation to block loop

This PR adds simulation to the block builder loop.

Updates the builder to use a SlotCalculator for managing slot timings.
Removes the tx and bundle poller from the block builder and instead leans into the actor model and feeds them in via channels to the simulation process.
This PR relies on the signet-sdk Sim Crate PR

Configuration

Deploying this PR will require additional environment variables to be set

CONCURRENCY_LIMIT to limit how many concurrent simulations the block builder runs.
START_TIMESTAMP to set the starting timestamp of the chain for anchoring slot calculation.

Testing

This code is still being tested before final merge.

The current test plan is to

Build a Docker image of the builder with this branch
Update environment variables to include the above values
Deploy the test image to Pecorino
Add a bundle to the cache and see if it is picked up
See what, if any, errors spring up
Land blocks and see that the block building rate is not decreased with new timing updates

- shifts the tx-poller to a more actor oriented approach by streaming transactions out of the cache. - transaction deduplication and validation is intended to be carried out by the simulator.

- improves poll interval handling - removes arcs - updates comments - adds file level comment

* fix: break loop on closure and improve tracing * refactor: break out the task future

- improves poll interval handling - cleans up and updates comments - removes arc usage

- adds init4 metrics - updates provider types to account for alloy changes

- align to alloy @ 0.12.6 - transfer off of zenith-rs and use signet-sdk instead

dylanlott · 2025-04-15T19:39:59Z

chore: refactor test utils #72
[feat] adds simulation to block loop #71 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

anna-carroll · 2025-04-25T16:17:52Z

src/tasks/tx_poller.rs

    }

-    /// polls the tx-pool for unique transactions and evicts expired transactions.
-    /// unique transactions that haven't been seen before are sent into the builder pipeline.
+    /// Polls the transaction cache for transactions.
    pub async fn check_tx_cache(&mut self) -> Result<Vec<TxEnvelope>, Error> {


FYI, this PR pulls this functionality into the SDK - but no need to block on it

Will address in a follow-up PR 👍

prestwich

we are going places 😎

Cargo.toml

bin/builder.rs

prestwich · 2025-04-28T12:39:37Z

bin/builder.rs


    let port = config.builder_port;
    let server = serve_builder_with_span(([0, 0, 0, 0], port), span);

    select! {
+        _ = tx_poller_jh => {
+            tracing::info!("tx_poller finished");


i miss is cooked :(

is cooked got cooked

src/config.rs

src/tasks/block.rs

src/tasks/bundler.rs

prestwich · 2025-04-28T13:03:42Z

src/tasks/tx_poller.rs

    pub config: BuilderConfig,
-    /// Reqwest client for fetching transactions from the tx-pool
+    /// Reqwest Client for fetching transactions from the cache.


we can replace this with TxCache after init4tech/signet-sdk#39

Will address in a follow up PR 👍

src/tasks/tx_poller.rs

- adds newlines after arguments and returns sections in comments - pulls several functions out into methods

…ng with env config

- refactors the rollup provider type from a FillProvider to a RootProvider - defines the alloy database provider type around this - updates relevant connection functions to use the new RuProvider type - renames Provider to HostProvider - renames WalletlessProvider to RuProvider

Evalir · 2025-04-29T13:28:30Z

src/consts.rs

@@ -0,0 +1,4 @@
+//! Constants used in the builder.


mega nit: let's just call this file constants.rs since that's the usual name elsewhere

EDIT: well just saw on the prev review that this was the suggestion. feel free to ignore

src/config.rs

Evalir · 2025-04-29T14:35:52Z

bin/builder.rs


    let port = config.builder_port;
    let server = serve_builder_with_span(([0, 0, 0, 0], port), span);

    select! {
+        _ = tx_poller_jh => {
+            tracing::info!("tx_poller finished");


is cooked got cooked

src/config.rs

prestwich · 2025-04-30T13:27:53Z

src/tasks/block.rs

+    ///
+    /// A `JoinHandle` for the basefee updater and a `JoinHandle` for the
+    /// cache handler.
+    pub fn spawn_cache_task(


nit: this actually spawns 2 tasks

prestwich · 2025-04-30T13:44:17Z

src/tasks/block.rs

+            AlloyDB::new(self.ru_provider.clone(), BlockId::from(latest));
+        let wrapped_db: WrapDatabaseAsync<AlloyDB<Ethereum, RuProvider>> =
+            WrapDatabaseAsync::new(alloy_db).unwrap_or_else(|| {
+                panic!("failed to acquire async alloy_db; check which runtime you're using")


this errorr only occurs on current thread runtimes, so we can actually rule it out entirely by setting the runtime flavor in the main macro link

prestwich · 2025-04-30T13:46:12Z

src/tasks/block.rs

+            db,
+            constants,
+            PecorinoCfg {},
+            NoopBlock,


can't use a noop-block here. need to load a header and modify in a way similar to what happens here:

https://github.com/init4tech/signet-sdk/blob/main/crates/rpc/src/ctx.rs#L500-L523

prestwich · 2025-04-30T13:48:16Z

src/tasks/block.rs

+
+        // Update the basefee on a per-block cadence
+        let basefee_jh =
+            tokio::spawn(async move { self.basefee_updater(Arc::clone(&basefee_price)).await });


this arc clone is unnecessary, as we already cloned above

prestwich · 2025-04-30T13:51:25Z

src/tasks/block.rs

+    async fn check_basefee(&self, price: &Arc<AtomicU64>) {
+        tracing::debug!("checking latest basefee");
+        let resp = self.ru_provider.get_block_by_number(Latest).await;
+        if let Err(e) = resp {


I think this fn could be cleaned up like this

resp.inspect_err(|e| tracing::debug!(...)); if let Ok(Some(resp)) = resp { ... }

prestwich · 2025-04-30T13:52:22Z

src/tasks/block.rs

+                    let _ = submit_sender.send(block);
+                }
+                Err(e) => {
+                    tracing::error!(err = %e, "failed to send block");


failed to send, or failed to build?

prestwich · 2025-04-30T13:54:11Z

src/tasks/block.rs

+    ///
+    /// An `Instant` representing the deadline.
+    pub fn calculate_deadline(&self) -> Instant {
+        let now = SystemTime::now();


i don't like that we're constantly mixing timestamps and Duration/Instant. it makes it very hard to reason about the logic. cc @anna-carroll can we add more convenience functions to SlotCalculator to make these easier? what are the common actions we want to take using a calculator?

prestwich · 2025-04-30T13:54:32Z

src/tasks/block.rs

-    // add a buffer to the beginning of the block slot.
-    fn secs_to_next_target(&self) -> u64 {
-        self.secs_to_next_slot() + self.config.target_slot_time
+        Instant::now().checked_add(Duration::from_secs(unix_seconds)).unwrap()


checked_add().unwrap() is a little redundant

prestwich · 2025-04-30T13:55:31Z

src/tasks/block.rs

-        in_progress_block.ingest_bundle(bundle);
+/// Configuration struct for Pecorino network values.
+#[derive(Debug, Clone)]
+pub struct PecorinoCfg {}


if no probs, use a simple structdef without the {}

pub struct PecorinoCfg;

also should impl Copy

prestwich · 2025-05-01T14:27:11Z

src/test_utils.rs

+pub fn setup_test_config() -> Result<BuilderConfig> {
+    let config = BuilderConfig {
+        host_chain_id: 17000,
+        ru_chain_id: 17001,


do these need to match the Pecorino chain id used below?

dylanlott and others added 15 commits April 3, 2025 09:45

updates the tx-poller to stream transactions

ef6b734

- shifts the tx-poller to a more actor oriented approach by streaming transactions out of the cache. - transaction deduplication and validation is intended to be carried out by the simulator.

refactors

1ad6e14

- improves poll interval handling - removes arcs - updates comments - adds file level comment

Fix: various updates to the tx poller (#67)

b5a5cf4

* fix: break loop on closure and improve tracing * refactor: break out the task future

updates bundler to streaming actor pattern

3117b85

refactors

96dce04

- improves poll interval handling - cleans up and updates comments - removes arc usage

fmt

5cfbdc9

fix: remove eq/partial eq

c4522ee

fix: various improvements to bundler tasks (#68)

01a11e6

chore: update to [email protected]

6374238

- adds init4 metrics - updates provider types to account for alloy changes

refactors to account for init4 bin base in builder craate

b1324b0

use signet-sdk

976dd03

- align to alloy @ 0.12.6 - transfer off of zenith-rs and use signet-sdk instead

fix: update bin-base import

b229996

nits: doc updates

bafe07f

lint: fmt

e0c79f7

feat: implements simulation for block building

785172b

dylanlott self-assigned this Apr 15, 2025

dylanlott added 5 commits April 15, 2025 13:40

lint: fmt

ff5cb31

cleanup

a142314

WIP: adding a block builder loop test

9d812ce

tests are passing now

bf75913

adds slot calculator to block builder

55b2193

dylanlott mentioned this pull request Apr 19, 2025

chore: refactor test utils #72

Closed

dylanlott added 2 commits April 18, 2025 20:37

refactors test utils across tasks

fd4bdcd

fmt

dd4b377

dylanlott force-pushed the dylan/sim-impl branch from ac4bbb1 to 47a2ed7 Compare April 19, 2025 03:09

fixup clippy and fmt

1ee991f

dylanlott force-pushed the dylan/sim-impl branch from 47a2ed7 to 1ee991f Compare April 19, 2025 03:13

dylanlott added 2 commits April 18, 2025 21:19

update cargo paths to sdk

50162b7

mark block builder test as integration test

14a0a62

dylanlott requested review from Evalir and prestwich as code owners April 24, 2025 17:48

ignore test spawn

70fdac1

anna-carroll reviewed Apr 25, 2025

View reviewed changes

prestwich requested changes Apr 28, 2025

View reviewed changes

dylanlott added 10 commits April 28, 2025 11:09

update signet-sim deps to main branch

6b59e39

try_join the setup of host, rollup, and sequencer services

09fc552

removes unused error from ConfigError

830f297

use AtomicU64 instead of RwLock in cache task handler

1ad4974

cleanup & refactors

b4c64da

- adds newlines after arguments and returns sections in comments - pulls several functions out into methods

removes unnecessary SimItem usage from cache updater

8300f03

makes tx-pool hold a Client reference for reuse

f27dde8

update comment

f63fe6b

sets concurrency limit by calling available_parallelism or overridi…

79977bd

…ng with env config

dylanlott requested a review from prestwich April 28, 2025 22:00

creates mod consts

8980e23

Evalir reviewed Apr 29, 2025

View reviewed changes

dylanlott added 2 commits April 29, 2025 11:05

pulls chain configs out into constants module

4e97d89

clippy + fmt

31d62d6

Evalir reviewed Apr 29, 2025

View reviewed changes

prestwich requested changes Apr 30, 2025

View reviewed changes

prestwich reviewed Apr 30, 2025

View reviewed changes

prestwich mentioned this pull request Apr 30, 2025

refactor: trim some dead code, simplify auth #73

Draft

prestwich reviewed May 1, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feat] adds simulation to block loop #71

[feat] adds simulation to block loop #71

dylanlott commented Apr 15, 2025 •

edited

Loading

dylanlott commented Apr 15, 2025 •

edited

Loading

anna-carroll Apr 25, 2025

dylanlott Apr 28, 2025

prestwich left a comment

prestwich Apr 28, 2025

Evalir Apr 29, 2025

prestwich Apr 28, 2025

dylanlott Apr 28, 2025

Evalir Apr 29, 2025 •

edited

Loading

Evalir Apr 29, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich Apr 30, 2025

prestwich May 1, 2025

[feat] adds simulation to block loop #71

Are you sure you want to change the base?

[feat] adds simulation to block loop #71

Conversation

dylanlott commented Apr 15, 2025 • edited Loading

[feat] adds simulation to block loop

Configuration

Testing

dylanlott commented Apr 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prestwich left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Evalir Apr 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dylanlott commented Apr 15, 2025 •

edited

Loading

dylanlott commented Apr 15, 2025 •

edited

Loading

Evalir Apr 29, 2025 •

edited

Loading