Merge pull request #6 from 0xPolygonZero/empty_trie

Empty trie

README.md

@@ -2,14 +2,132 @@
 
 Similar to [`eth-proof`](https://github.com/wborgeaud/eth-proof) but for transaction proofs.
 
+## Quick Start
+
+There are two ways to run this prover. The simplest way to get started is
+to use the `in-memory` runtime of
+[Paladin](https://github.com/0xPolygonZero/paladin). This requires
+very little setup, but it's not really suitable for a large scale
+test. The other method for testing the prover is to leverage an
+[AMQP](https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol)
+like [RabbitMQ](https://en.wikipedia.org/wiki/RabbitMQ) to distribute
+workload over many workers.
+
+### Setup
+
+Before running the prover, you'll need to compile the
+application. This command should do the trick:
+
+```bash
+env RUSTFLAGS='-C target-cpu=native' cargo build --release
+```
+
+You should end up with two binaries in your `target/release`
+folder. One is called `worker` and the other is `leader`. Typically,
+we'll install these somewhere in our `$PATH` for convenience.
+
+Once you have application available, you'll need to create a block
+[witness](https://nmohnblatt.github.io/zk-jargon-decoder/definitions/witness.html)
+which essentially serves as the input for the prover. Assuming you've
+deployed the `leader` binary, you should be able to generate a witness
+like this:
+
+```bash
+paladin-leader rpc -u $RPC_URL -t 0x2f0faea6778845b02f9faf84e7e911ef12c287ce7deb924c5925f3626c77906e > 0x2f0faea6778845b02f9faf84e7e911ef12c287ce7deb924c5925f3626c77906e.json
+```
+
+You'll need access to an Ethereum RPC in order to run this
+command. The input argument is a transaction hash and in particular it
+is the *last* transaction has in the block.
+
+Once you've successfully generated a witness, you're ready to start
+proving either with the `in-memory` runtime or the `amqp` runtime.
+
+### In Memory Proving
+
+Running the prover with the `in-memory` flag requires no setup. You
+can attempt to generate a proof with a command like this.
+
+```bash
+env RUST_MIN_STACK=33554432 \
+ARITHMETIC_CIRCUIT_SIZE="15..28" \
+BYTE_PACKING_CIRCUIT_SIZE="9..28" \
+CPU_CIRCUIT_SIZE="12..28" \
+KECCAK_CIRCUIT_SIZE="14..28" \
+KECCAK_SPONGE_CIRCUIT_SIZE="9..28" \
+LOGIC_CIRCUIT_SIZE="12..28" \
+MEMORY_CIRCUIT_SIZE="17..30" \
+paladin-leader prove \
+--runtime in-memory \
+--num-workers 1 \
+--input-witness 0x2f0faea6778845b02f9faf84e7e911ef12c287ce7deb924c5925f3626c77906e.json
+```
+
+The circuit parameters here are meant to be compatible with virtually
+all Ethereum blocks. This will create a block proof from an input
+state root of the preceding block. You can adjust the `--num-workers`
+flag based on the number of available compute resources. As a rule of
+thumb, you'd probably want at least 8 cores per worker. It's also
+worth noting that you'll probably want at least 40GB of physical
+memory to run the prover.
+
+### AMQP Proving
+
+Proving in a distributed compute environment depends on an AMQP
+server. We're not going to cover the setup of RabbitMQ, but assuming
+you have something like that available you can run a "leader" which
+distribute proving tasks to a collection of "workers" which actually
+do the proving work.
+
+In order to run the workers, you'll use a command like:
+
+```bash
+env RUST_MIN_STACK=33554432 \
+ARITHMETIC_CIRCUIT_SIZE="15..28" \
+BYTE_PACKING_CIRCUIT_SIZE="9..28" \
+CPU_CIRCUIT_SIZE="12..28" \
+KECCAK_CIRCUIT_SIZE="14..28" \
+KECCAK_SPONGE_CIRCUIT_SIZE="9..28" \
+LOGIC_CIRCUIT_SIZE="12..28" \
+MEMORY_CIRCUIT_SIZE="17..30" \
+paladin-worker --runtime amqp --amqp-uri=amqp://localhost:5672
+```
+
+This will start the worker and have it await tasks. Depending on the
+size of your machine, you may be able to run several workers on the
+same operating system. An example [systemd
+service](./deploy/[email protected]) is included. Once that
+service is installed, you could enable and start 16 workers on the
+same VM like this:
+
+```bash
+seq 0 15 | xargs -I xxx systemctl enable paladin-worker@xxx
+seq 0 15 | xargs -I xxx systemctl start paladin-worker@xxx
+```
+
+Now that you have your pool of paladin workers, you can start proving
+with a command like this:
+
+```bash
+paladin-leader prove \
+--runtime amqp \
+--amqp-uri=amqp://localhost:5672 \
+--input-witness 0x2f0faea6778845b02f9faf84e7e911ef12c287ce7deb924c5925f3626c77906e.json
+```
+
+This command will run the same way as the `in-memory` mode except that
+the leader itself isn't doing the work. The separate worker processes
+are doing the heavy lifting.
+
+
 ## License
-Copyright (c) 2023 PT Services DMCC
 
 Licensed under either of:
+
 * Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
 * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
 
-at your option. 
+at your option.
 
 The SPDX license identifier for this project is `MIT OR Apache-2.0`.
 

common/src/prover_state/persistence.rs

@@ -59,6 +59,7 @@ pub fn to_disk(circuits: &AllRecursiveCircuits, circuit_config: &CircuitConfig)
     let file = OpenOptions::new()
         .write(true)
         .create(true)
+        .truncate(false)
         .open(disk_path(circuit_config));
 
     let mut file = match file {

deploy/paladin-worker@.service

@@ -0,0 +1,99 @@
+[Unit]
+Description=Paladin Prover Service
+Documentation=https://github.com/0xPolygonZero/eth-tx-proof/
+
+# Bring this up after the network is online
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+ExecStart=paladin-worker --runtime amqp --amqp-uri=amqp://localhost:5672
+
+MemoryHigh=750G
+MemoryMax=800G
+MemorySwapMax=2000G
+
+Restart=on-failure
+RestartSec=5s
+
+Type=simple
+
+User=root
+Group=root
+
+Environment=RUST_BACKTRACE=1
+Environment=RUST_LOG=info
+Environment=RAYON_NUM_THREADS=180
+Environment=RUST_MIN_STACK=33554432
+Environment=ARITHMETIC_CIRCUIT_SIZE="15..28"
+Environment=BYTE_PACKING_CIRCUIT_SIZE="9..28"
+Environment=CPU_CIRCUIT_SIZE="12..28"
+Environment=KECCAK_CIRCUIT_SIZE="14..28"
+Environment=KECCAK_SPONGE_CIRCUIT_SIZE="9..28"
+Environment=LOGIC_CIRCUIT_SIZE="12..28"
+Environment=MEMORY_CIRCUIT_SIZE="17..30"
+
+TimeoutStartSec=infinity
+TimeoutStopSec=600
+
+RuntimeDirectory=paladin
+RuntimeDirectoryMode=0700
+
+ConfigurationDirectory=paladin
+ConfigurationDirectoryMode=0700
+
+StateDirectory=paladin
+StateDirectoryMode=0700
+
+WorkingDirectory=/var/lib/paladin
+
+# Hardening measures
+# https://www.linuxjournal.com/content/systemd-service-strengthening
+# sudo systemd-analyze security
+# systemd-analyze syscall-filter
+####################
+
+# Provide a private /tmp and /var/tmp.
+PrivateTmp=true
+
+# Mount /usr, /boot/ and /etc read-only for the process.
+ProtectSystem=full
+
+# Deny access to /home, /root and /run/user
+ProtectHome=true
+
+# Disallow the process and all of its children to gain
+# new privileges through execve().
+NoNewPrivileges=true
+
+# Use a new /dev namespace only populated with API pseudo devices
+# such as /dev/null, /dev/zero and /dev/random.
+PrivateDevices=true
+
+# Deny the creation of writable and executable memory mappings.
+MemoryDenyWriteExecute=true
+# MemoryDenyWriteExecute=false
+
+# Deny any ability to create namespaces. Should not be needed
+RestrictNamespaces=true
+
+# Restrict any kind of special capabilities
+CapabilityBoundingSet=
+
+# Allow minimal system calls for IO (filesystem network) and basic systemctl operations
+SystemCallFilter=@signal @network-io @ipc @file-system @chown @system-service
+
+# Access to  /sys/fs/cgroup/ should not be needed
+ProtectControlGroups=true
+
+# We don't need access to special file systems or extra kernel modules to work
+ProtectKernelModules=true
+
+# Access to proc/sys/, /sys/, /proc/sysrq-trigger, /proc/latency_stats, /proc/acpi, /proc/timer_stats, /proc/fs and /proc/irq is not needed
+ProtectKernelTunables=true
+
+# From the docsk "As the SUID/SGID bits are mechanisms to elevate privileges, and allow users to acquire the identity of other users, it is recommended to restrict creation of SUID/SGID files to the few programs that actually require them"
+RestrictSUIDSGID=true
+
+[Install]
+WantedBy=multi-user.target

leader/src/lib.rs

@@ -117,6 +117,30 @@ pub async fn get_block_metadata(
     ))
 }
 
+pub async fn get_block_hashes(block_number: U64, provider: &Provider<Http>) -> Result<BlockHashes> {
+    let mut prev_hashes = vec![H256::zero(); 256];
+    let cur_hash = provider
+        .get_block(block_number)
+        .await?
+        .ok_or_else(|| anyhow!("Block not found. Block number: {}", block_number))?
+        .hash
+        .unwrap();
+    for i in 1..=256 {
+        let hash = provider
+            .get_block(block_number - i)
+            .await?
+            .ok_or_else(|| anyhow!("Block not found. Block number: {}", block_number - i))?
+            .hash
+            .unwrap();
+        prev_hashes[256 - i] = hash;
+    }
+
+    Ok(BlockHashes {
+        prev_hashes,
+        cur_hash,
+    })
+}
+
 pub async fn gather_witness(tx: TxHash, provider: &Provider<Http>) -> Result<Vec<TxnProofGenIR>> {
     let tx = provider
         .get_transaction(tx)
@@ -306,6 +330,8 @@ pub async fn gather_witness(tx: TxHash, provider: &Provider<Http>) -> Result<Vec
     } else {
         vec![]
     };
+    // Block hashes
+    let block_hashes = get_block_hashes(block_number.into(), provider).await?;
 
     let mut storage_mpts: HashMap<_, _> = storage_mpts
         .iter()
@@ -391,10 +417,7 @@ pub async fn gather_witness(tx: TxHash, provider: &Provider<Http>) -> Result<Vec
             withdrawals,
             contract_code: contract_codes.clone(),
             block_metadata: block_metadata.clone(),
-            block_hashes: BlockHashes {
-                prev_hashes: vec![H256::zero(); 256], // TODO
-                cur_hash: H256::zero(),               // TODO
-            },
+            block_hashes: block_hashes.clone(),
             gas_used_before: gas_used,
             gas_used_after: gas_used + receipt.gas_used.unwrap(),
             checkpoint_state_trie_root: prev_block.state_root, // TODO: make it configurable

leader/src/mpt.rs

@@ -21,6 +21,11 @@ pub struct Mpt {
     pub root: H256,
 }
 
+const EMPTY_TRIE_HASH: H256 = H256([
+    86, 232, 31, 23, 27, 204, 85, 166, 255, 131, 69, 230, 146, 192, 248, 110, 91, 72, 224, 27, 153,
+    108, 173, 192, 1, 98, 47, 181, 227, 99, 180, 33,
+]);
+
 impl Mpt {
     pub fn new() -> Self {
         Self {
@@ -30,8 +35,14 @@ impl Mpt {
     }
 
     pub fn to_partial_trie(&self) -> HashedPartialTrie {
-        self.to_partial_trie_helper(self.root)
+        let trie = self.to_partial_trie_helper(self.root);
+        if trie == Node::Hash(EMPTY_TRIE_HASH).into() {
+            Node::Empty.into()
+        } else {
+            trie
+        }
     }
+
     fn to_partial_trie_helper(&self, root: H256) -> HashedPartialTrie {
         let node = self.mpt.get(&root);
         let data = if let Some(mpt_node) = node {