Closes: EspressoSystems/zkrollup-integration/issues/4

tl;dr

1. This PR merely lays the foundation for an sp1-prover Rust component which may be able to replace the existing zkevm-prover in the future. The new sp1-prover component is comprised of three (mocked) services: aggregator, executor_service and hashdb_service.
2. Testing with the new sp1-prover component can be done in 2 different configurations:
   1. make run-sp1
      This runs the full node test but with the new sp1-prover in addition to the existing zkevm-prover. The new sp1-prover will announce itself to the aggregator and will eventually receive proof requests. sp1-prover can be configured to emulate the zkevm-prover’s aggregator-client-mock.cpp using data/mocked_data.json as mock data with which verification will succeed. Or using other data (e.g. data/mocked_sp1_data.json) which will cause the demo (aggregator) to eventually fail in verification. See more details below in: "Running e2e test with additional sp1 prover".
   2. make run-sp1-only
      This configuration is for exploring the work needed to completely replace the zkevm-prover including execturor_service and hashdb_service. Upon startup the synchronizer needs to connect to both executor and hashdb to be able to make progress.
      For now, only a few RPC calls are stubbed for these services and the rest remains unimplemented. See more details below: "Running e2e test with sp1-prover only".
3. The overall goal with make run-sp1 (and this PR) is to unblock any potential contracts work further up the stack (Aggregator/EthTxManager/L1). Fully replacing the zkevm-prover (blackbox) seems like a very heavy lift. This is mainly due to the prover’s responsibilities as executor and hashdb. It might be worth investigating if we can extract the “prover”-part of the zkevm-prover (communicating with the aggregator) while keeping the rest of the logic (executor/hashdb) intact. However, the intricacies of the prover logic make this seem non-trivial. Finally, since the purpose of this PR is mainly to unblock other work and explore different design strategies for replacing the prover, the code is unpolished and lags basic error handling and logging infra.

Common Setup

Make sure that cargo is able to cross compile targeting musl libc (for MacOS/M1 see fx https://github.com/FiloSottile/homebrew-musl-cross) and add the correct linker. Fx:

[target.x86_64-unknown-linux-musl]
linker = "x86_64-linux-musl-gcc"

Build the binary:

cdk-validium-node % cd sp1-prover
sp1-prover % cargo build --release --target=x86_64-unknown-linux-musl

Build the docker images (sp1-prover and zkevm-node):

cdk-validium-node % make build-docker-sp1

Running e2e test with additional sp1-prover

(Make sure that the Common Setup has been completed)

In order to test that the verification indeed fails when we replace the proof, we do the following:

cdk-validium-node % cd test
test % make run-sp1

This will run the e2e test with both sp1-prover and zkevm-prover. The sp1-prover will connect to the aggregator and await any proof requests. We can check if sp1-prover has been asked for a final proof:

% docker logs -f sp1-prover |& grep -e "GenFinalProofRequest" -e "GetProofRequest"
Received request: GetProofRequest { id: "aNMsMmQDUOh3KBOuCS9A0mvFIaORcEx2bzhru", timeout: 0 }
Received request: GenFinalProofRequest { recursive_proof: "88888670604050723159190639550237390237901487387303122609079617855313706601738", aggregator_addr: "0x70997970c51812dc3a010c7d01b50e0d17dc79c8" }
Received request: GetProofRequest { id: "RZkQ5d14SrB6hBIxTLGzwM2iCguw8qzl9uuNl", timeout: 0 }

To check that the zkevm-aggregator has received the proof and it has been correctly verified, we check:

test % docker logs -f zkevm-aggregator |& grep -e "Final proof verified"
2024-07-25T13:10:12.783Z	INFO	aggregator/aggregator.go:1300	Final proof verified	{"pid": 1, "version": "923b75a8", "txId": "proof-from-1-to-1", "batches": "1-1"}

Making the verification fail:

We can make the verification fail by submitting a different proof to the aggregator.

Change line 11 in sp1-prover/src/aggregator from:

static MOCKED_DATA: &str = include_str!("data/mocked_data.json");

to

static MOCKED_DATA: &str = include_str!("data/mocked_sp1_data.json");

such that the aggregator fetches the mock data from mocked_sp1_data.json instead.

Rebuild binary and docker image (see: Common Setup section).

Run the demo again and check that the verification fails:

test % make run-sp1
test % docker logs -f zkevm-aggregator |& grep -e "ERROR"
2024-07-25T13:34:53.258Z	ERROR	etherman/etherman.go:1027	error converting proof. Error: invalid proof length. Length: 1730, Proof:[...]

Running e2e test with sp1-prover only

Todo
Postponing further description of this configuration because (as of now) replacing the full prover (incl. executor and hashdb) seems infeasible to do within a reasonable timeline.

This PR does not:

Let nix support the cross compilation through the use of cross-shell.nix. For now, the cross compilation can be done outside of the nix environment. This should be fixed in the near future to avoid inconsistencies between environments.