Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TxPool v2 General architecture #2162

Merged
merged 83 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from 72 commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
2e07d37
Add skeleton txpool v2
AurelienFT Sep 4, 2024
4cbb11c
Add basic interfaces and retrieve some functionnalities of the old pool.
AurelienFT Sep 5, 2024
bef4aff
Add all machinery to make the tests
AurelienFT Sep 5, 2024
552af83
Add some logic that needs to be moved behind abstraction
AurelienFT Sep 6, 2024
8d39499
Add all storages for transactions and clean code a bit
AurelienFT Sep 9, 2024
3d1a1bb
Merge branch 'master' into txpool-v2
AurelienFT Sep 9, 2024
3d65076
Fix compilation errors and add selection mecanism
AurelienFT Sep 9, 2024
19830cb
Start thinking about traits
AurelienFT Sep 9, 2024
795567e
Add all abstractions in txpool to clean code
AurelienFT Sep 10, 2024
67afb9d
Split all logic across implementations and uncomment half of the tests
AurelienFT Sep 10, 2024
b956e2c
Add all tests
AurelienFT Sep 10, 2024
9ed577e
Fmt + toml lint + spellcheck
AurelienFT Sep 10, 2024
29ec06d
Resolves most of the TODO, remove unwrap and add test
AurelienFT Sep 11, 2024
9a22983
Document and allow warnings on dead code/unused
AurelienFT Sep 11, 2024
ca38a25
Merge branch 'master' into txpool-v2
AurelienFT Sep 11, 2024
fe9e995
fix fuel-core-type usage in ttxpool v2
AurelienFT Sep 11, 2024
34c7e07
Remove usage of txpool v2 in fuel-core
AurelienFT Sep 11, 2024
4576797
fmt fix
AurelienFT Sep 11, 2024
c8e86ec
Don't allow dependencies on output and change and specify todos
AurelienFT Sep 11, 2024
16a7e8f
Rework dependency between components
AurelienFT Sep 12, 2024
60ada65
Simplify usage of the db trait
AurelienFT Sep 12, 2024
0cf8835
update graph cumultative gas and tip
AurelienFT Sep 12, 2024
b328bc8
Docs fixes and config renaming
AurelienFT Sep 12, 2024
0d69dc1
fmt
AurelienFT Sep 12, 2024
75a9180
Use persistant storage instead of db and reuse a view provider
AurelienFT Sep 12, 2024
87e5027
Fix comments in tests
AurelienFT Sep 12, 2024
4e3e3ce
Update components trait to be more flexible.
AurelienFT Sep 13, 2024
826f13e
Update tests and make the insert function more readable
AurelienFT Sep 13, 2024
bafbf75
remove over constraint on trait and rename a test.
AurelienFT Sep 16, 2024
ed2302a
Merge branch 'master' into txpool-v2
AurelienFT Sep 16, 2024
d4ebf3c
Change some documentation and remove a lot of dead code.
AurelienFT Sep 18, 2024
45d5296
Merge branch 'master' into txpool-v2
AurelienFT Sep 18, 2024
1c1e5db
Update cargo lock
AurelienFT Sep 18, 2024
6e5c7f8
Merge branch 'master' into txpool-v2
AurelienFT Sep 19, 2024
2a5aabd
Merge branch 'master' into txpool-v2
AurelienFT Sep 19, 2024
e78544f
Fix changelog
AurelienFT Sep 19, 2024
dd7c095
Merge branch 'master' into txpool-v2
AurelienFT Sep 20, 2024
14b9b53
Update crates/services/txpool_v2/src/lib.rs
AurelienFT Sep 20, 2024
68b1157
Remove unused parameters and secure the collisions subfields
AurelienFT Sep 20, 2024
1074e5a
Rename functions add checks, and fix clippy
AurelienFT Sep 20, 2024
f89174f
fmt
AurelienFT Sep 20, 2024
3ddea7e
Change max txs in chain rules
AurelienFT Sep 20, 2024
a5ff2ab
fmt
AurelienFT Sep 20, 2024
dd9ef6e
Moved collisions removal after checks
AurelienFT Sep 20, 2024
f0476e2
Add some caches to avoid more iteration and fix collission taken as d…
AurelienFT Sep 20, 2024
2f5a199
Merge branch 'master' into txpool-v2
AurelienFT Sep 20, 2024
ade8569
Split two functions and optimize iterator returns
AurelienFT Sep 21, 2024
6beb83d
Merge branch 'master' into txpool-v2
AurelienFT Sep 21, 2024
3dde832
elllude lifetime
AurelienFT Sep 23, 2024
f5ef031
update review comments
AurelienFT Sep 26, 2024
2bd0382
Merge branch 'master' into txpool-v2
AurelienFT Sep 26, 2024
e9c78fc
remove wrong debug assert
AurelienFT Sep 26, 2024
f955ee5
Merge branch 'txpool-v2' of github.com:FuelLabs/fuel-core into txpool-v2
AurelienFT Sep 26, 2024
3be0fc7
Remove unused error in storage and change collisions to hashset.
AurelienFT Sep 27, 2024
24cc034
change graph information stored in storedata
AurelienFT Sep 27, 2024
aa8cb24
Fix remove of a cache done in a case where it's not relevant
AurelienFT Sep 27, 2024
0bbdaf5
Make sure to edit all dependencies when we remove a dependent subtree.
AurelienFT Sep 27, 2024
c128c75
Txpool v2 insertion (#2193)
AurelienFT Sep 27, 2024
aeb2935
Minimized the number of `Result` in the code in places where we can't…
xgreenx Sep 28, 2024
8a746bf
Added fuzzer for TxPoolV1 and TxPoolV2 to create dependent graphs. It…
xgreenx Sep 28, 2024
0fcb185
Applied comments from the review
xgreenx Sep 29, 2024
88cec23
Make clippy happy
xgreenx Sep 29, 2024
47ebb4c
Moved the logic of collision and depdednencies verificaiton to the po…
xgreenx Sep 29, 2024
eed0d13
Revert rocksdb
xgreenx Sep 29, 2024
d88a016
Fix tests
xgreenx Sep 29, 2024
2fd7c15
Reuse direct dedpednencies.
xgreenx Sep 29, 2024
99da36b
Merge branch 'refs/heads/master' into txpool-v2
xgreenx Sep 29, 2024
a540eda
Reject transactions that creates diamond dependencies. It allows simp…
xgreenx Sep 30, 2024
b25b832
Make limits less strict for fuzzer. It is okay since we don't need to…
xgreenx Sep 30, 2024
acf0ab5
add checks cfg on unused code
AurelienFT Oct 1, 2024
6df300a
Applied comments from the PR
xgreenx Oct 1, 2024
d16424c
Added a comment for stabiltiy test
xgreenx Oct 1, 2024
6fe6d98
Merge branch 'master' into txpool-v2
xgreenx Oct 1, 2024
5def1d3
update tests following review
AurelienFT Oct 1, 2024
3b7fc49
remove unused and change order verif
AurelienFT Oct 1, 2024
48090ea
fix tests
AurelienFT Oct 2, 2024
1cd89e4
Merge branch 'master' into txpool-v2
AurelienFT Oct 2, 2024
1a96d77
update insert return type
AurelienFT Oct 2, 2024
de93d33
address tests reviews
AurelienFT Oct 3, 2024
55e770f
Merge branch 'master' into txpool-v2
xgreenx Oct 3, 2024
e86ddad
Update storage index in pool to be a template type and add a warn for…
AurelienFT Oct 3, 2024
414a8c7
Merge branch 'txpool-v2' of github.com:FuelLabs/fuel-core into txpool-v2
AurelienFT Oct 3, 2024
b7e93c0
remove copy bound
AurelienFT Oct 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- [2131](https://github.com/FuelLabs/fuel-core/pull/2131): Add flow in TxPool in order to ask to newly connected peers to share their transaction pool
- [2182](https://github.com/FuelLabs/fuel-core/pull/2151): Limit number of transactions that can be fetched via TxSource::next
- [2189](https://github.com/FuelLabs/fuel-core/pull/2151): Select next DA height to never include more than u16::MAX -1 transactions from L1.
- [2162](https://github.com/FuelLabs/fuel-core/pull/2162): Pool structure with dependencies, etc.. for the next transaction pool module.
- [2193](https://github.com/FuelLabs/fuel-core/pull/2193): Insertion in PoolV2 and tests refactoring


### Changed
Expand Down
22 changes: 22 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ members = [
"crates/services/relayer",
"crates/services/sync",
"crates/services/txpool",
"crates/services/txpool_v2",
"crates/services/upgradable-executor",
"crates/services/upgradable-executor/wasm-executor",
"crates/storage",
Expand Down Expand Up @@ -75,6 +76,7 @@ fuel-core-producer = { version = "0.36.0", path = "./crates/services/producer" }
fuel-core-relayer = { version = "0.36.0", path = "./crates/services/relayer" }
fuel-core-sync = { version = "0.36.0", path = "./crates/services/sync" }
fuel-core-txpool = { version = "0.36.0", path = "./crates/services/txpool" }
fuel-core-txpool-v2 = { version = "0.36.0", path = "./crates/services/txpool_v2" }
fuel-core-storage = { version = "0.36.0", path = "./crates/storage", default-features = false }
fuel-core-trace = { version = "0.36.0", path = "./crates/trace" }
fuel-core-types = { version = "0.36.0", path = "./crates/types", default-features = false }
Expand Down
15 changes: 9 additions & 6 deletions crates/services/txpool/src/containers/dependency.rs
Original file line number Diff line number Diff line change
Expand Up @@ -294,12 +294,15 @@ impl Dependency {
}
} else {
// tx output is in pool
let output_tx = txs.get(utxo_id.tx_id()).unwrap();
let output =
&output_tx.outputs()[utxo_id.output_index() as usize];
Self::check_if_coin_input_can_spend_output(
output, input, false,
)?;
let output_tx = txs.get(utxo_id.tx_id());
let output = output_tx.and_then(|tx| {
tx.outputs().get(utxo_id.output_index() as usize)
});
if let Some(output) = output {
Self::check_if_coin_input_can_spend_output(
output, input, false,
)?;
}
AurelienFT marked this conversation as resolved.
Show resolved Hide resolved
}

collided.push(*spend_by);
Expand Down
7 changes: 5 additions & 2 deletions crates/services/txpool/src/transaction_selector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,10 @@ mod tests {
checked_transaction::builder::TransactionBuilderExt,
SecretKey,
},
services::txpool::PoolTransaction,
services::txpool::{
Metadata,
PoolTransaction,
},
};
use itertools::Itertools;
use std::sync::Arc;
Expand Down Expand Up @@ -157,7 +160,7 @@ mod tests {
// so it doesn't need to compute valid sigs for tests
PoolTransaction::Script(
script.finalize_checked_basic(Default::default()),
ConsensusParametersVersion::MIN,
Metadata::new(ConsensusParametersVersion::MIN),
)
})
.map(Arc::new)
Expand Down
22 changes: 12 additions & 10 deletions crates/services/txpool/src/txpool.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,10 @@ use fuel_core_types::{
checked_transaction::CheckPredicateParams,
interpreter::Memory,
},
services::executor::TransactionExecutionStatus,
services::{
executor::TransactionExecutionStatus,
txpool::Metadata,
},
};
use std::{
collections::HashMap,
Expand Down Expand Up @@ -343,26 +346,25 @@ where
tx: Checked<Transaction>,
) -> Result<InsertionResult, Error> {
let view = self.database.latest_view().unwrap();
self.insert_inner(tx, ConsensusParametersVersion::MIN, &view)
self.insert_inner(tx, Metadata::new(ConsensusParametersVersion::MIN), &view)
}

#[tracing::instrument(level = "debug", skip_all, fields(tx_id = %tx.id()), ret, err)]
// this is atomic operation. Return removed(pushed out/replaced) transactions
fn insert_inner(
&mut self,
tx: Checked<Transaction>,
version: ConsensusParametersVersion,
metadata: Metadata,
view: &View,
) -> Result<InsertionResult, Error> {
let tx: CheckedTransaction = tx.into();

let tx = Arc::new(match tx {
CheckedTransaction::Script(tx) => PoolTransaction::Script(tx, version),
CheckedTransaction::Create(tx) => PoolTransaction::Create(tx, version),
CheckedTransaction::Script(tx) => PoolTransaction::Script(tx, metadata),
CheckedTransaction::Create(tx) => PoolTransaction::Create(tx, metadata),
CheckedTransaction::Mint(_) => return Err(Error::MintIsDisallowed),
CheckedTransaction::Upgrade(tx) => PoolTransaction::Upgrade(tx, version),
CheckedTransaction::Upload(tx) => PoolTransaction::Upload(tx, version),
CheckedTransaction::Blob(tx) => PoolTransaction::Blob(tx, version),
CheckedTransaction::Upgrade(tx) => PoolTransaction::Upgrade(tx, metadata),
CheckedTransaction::Upload(tx) => PoolTransaction::Upload(tx, metadata),
CheckedTransaction::Blob(tx) => PoolTransaction::Blob(tx, metadata),
});

self.check_blacklisting(tx.as_ref())?;
Expand Down Expand Up @@ -450,7 +452,7 @@ where
};

for tx in txs.into_iter() {
res.push(self.insert_inner(tx, version, &view));
res.push(self.insert_inner(tx, Metadata::new(version), &view));
}

// announce to subscribers
Expand Down
39 changes: 39 additions & 0 deletions crates/services/txpool_v2/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
[package]
name = "fuel-core-txpool-v2"
version = { workspace = true }
authors = { workspace = true }
categories = ["cryptography::cryptocurrencies"]
edition = { workspace = true }
homepage = { workspace = true }
keywords = ["blockchain", "cryptocurrencies", "fuel-vm", "vm"]
license = { workspace = true }
repository = { workspace = true }
description = "Transaction pool v2"
publish = false
AurelienFT marked this conversation as resolved.
Show resolved Hide resolved

[dependencies]
anyhow = { workspace = true }
async-trait = { workspace = true }
derive_more = { workspace = true }
fuel-core-services = { workspace = true }
fuel-core-storage = { workspace = true, features = ["std"] }
fuel-core-types = { workspace = true, features = ["test-helpers"] }
futures = { workspace = true }
mockall = { workspace = true, optional = true }
num-rational = { workspace = true }
parking_lot = { workspace = true }
petgraph = "0.6.5"
rayon = { workspace = true }
tokio = { workspace = true, default-features = false, features = ["sync"] }
tracing = { workspace = true }

[dev-dependencies]
fuel-core-trace = { workspace = true }
rand = { workspace = true }

[features]
test-helpers = [
"dep:mockall",
"fuel-core-types/test-helpers",
"fuel-core-storage/test-helpers",
]
Loading
Loading