Tcp signer setup with docker

.dockerignore

@@ -0,0 +1,4 @@
+.env
+logs/
+bitcoin-data/
+node_modules/

.env-example

@@ -1,5 +1,8 @@
 POWPEG_NODE_JAR_PATH=/Users/<your_user>/repos/powpeg-node/build/libs/federate-node-SNAPSHOT-<version>all.jar
+
+# Change to ./config/regtest-key-files-and-hsms to also tests with the tcp signer
 CONFIG_FILE_PATH=./config/regtest-all-keyfiles
+
 LOG_HOME=/Users/<your_user>/config/logs-config
 BITCOIND_BIN_PATH=/Users/<your_user>/bitcoind/bin/bitcoind
 JAVA_BIN_PATH=/Library/Java/JavaVirtualMachines/adoptopenjdk-17.jdk/Contents/Home/bin/java
@@ -24,3 +27,9 @@ WAIT_FOR_BLOCK_ATTEMPT_TIME_MILLIS=
 # `80` recommended for most machines with enough resources. `160`, `250` or more for machine with limited resources.
 # Adjust as needed, starting with low values so the tests run as fast as they can.
 WAIT_FOR_BLOCK_MAX_ATTEMPTS=
+
+# Use to know how to execute the tcpsigner. So far these tests are designed to run on macOS or Ubuntu. If no option is specified, then the tcpsigner will be executed for Ubuntu.
+EXEC_ENV=MACOS
+
+# Set it to false if you want to keep the container after the tests are executed. This is useful for debugging purposes. You can use the `docker exec` command to enter the container and check the logs or any other files.
+DOCKER_REMOVE_CONTAINER_AFTER_EXECUTION=true

.env.docker

@@ -0,0 +1,8 @@
+POWPEG_NODE_JAR_PATH=/rits/federate-node.jar
+
+# Change to /rits/config/regtest-key-files-and-hsms to also tests with the tcp signer
+CONFIG_FILE_PATH=/rits/config/regtest-all-keyfiles
+
+LOG_HOME=/rits/logs
+
+BITCOIN_DATA_DIR=/rits/bitcoin-data

.github/workflows/ci.yml

@@ -35,10 +35,7 @@ jobs:
       - name: Checkout
         id: checkout
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-
-      - name: Prepare Docker build context
-        run: cp config/regtest-all-keyfiles.js container-action/rit-local-configs/
-
+
       - name: Docker meta
         id: meta
         uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1

.gitignore

@@ -7,3 +7,13 @@ logs
 .env
 .DS_Store
 .idea/
+_internal/
+manager-tcp
+bitcoin-data/
+key.json
+tcpsigner-manager.log
+tcpsigner.log
+*.xml
+!base-logback-config.xml
+logs/
+federate-node.jar

Dockerfile

@@ -0,0 +1,86 @@
+FROM ubuntu:24.04@sha256:2e863c44b718727c860746568e1d54afd13b2fa71b160f5cd9058fc436217b30 AS builder
+
+LABEL Description="Custom RSK node image to execute Rootstock Integration Tests"
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends \
+        ca-certificates \
+        curl \
+        git \
+        gnupg2 \
+        mocha \
+        wget \
+        build-essential \
+        python3 \
+    && apt clean
+
+RUN apt-get update && apt-get install -y libc6
+
+# -- nodeJs ---------------------------------------------------------
+ENV NODE_VERSION v18.20.2
+RUN mkdir -p /usr/local/nvm
+ENV NVM_DIR /usr/local/nvm
+
+RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash \
+    && . $NVM_DIR/nvm.sh \
+    && nvm install $NODE_VERSION \
+    && nvm alias default $NODE_VERSION \
+    && nvm use default
+
+ENV NODE_PATH $NVM_DIR/$NODE_VERSION/lib/node_modules
+ENV PATH $NVM_DIR/versions/node/$NODE_VERSION/bin:$PATH
+
+# -- java ---------------------------------------------------------
+ENV JAVA_VERSION 17
+
+RUN apt-get update \
+    && apt-get -y install "openjdk-$JAVA_VERSION-jdk"
+
+ENV JAVA_HOME="/usr/lib/jvm/java-$JAVA_VERSION-openjdk-amd64"
+
+# -- bitcoind ---------------------------------------------------------
+ENV BITCOIN_VERSION 0.18.1
+
+RUN cd /tmp \
+    && wget https://bitcoincore.org/bin/bitcoin-core-${BITCOIN_VERSION}/bitcoin-${BITCOIN_VERSION}-x86_64-linux-gnu.tar.gz \
+    && tar -xzvf bitcoin-${BITCOIN_VERSION}-x86_64-linux-gnu.tar.gz -C /opt \
+    && mv /opt/bitcoin-${BITCOIN_VERSION} /opt/bitcoin \
+    && rm -v /opt/bitcoin/bin/test_bitcoin /opt/bitcoin/bin/bitcoin-qt \
+    && ln -sv /opt/bitcoin/bin/* /usr/local/bin
+
+RUN apt-get update && \
+    apt-get install -y procps libsecp256k1-dev && \
+    rm -rf /var/lib/apt/lists/*
+
+# Set work directory for Node.js
+WORKDIR /rits
+
+RUN mkdir /rits/bitcoin-data
+
+# Copy Node.js dependencies and install
+COPY package*.json ./
+RUN npm install
+
+# Copy the rest of the Node.js project
+COPY . .
+
+# Overriding .env file with docker specific values
+COPY .env.docker /rits/.env
+
+RUN chmod +x /rits/configure.sh
+RUN /rits/configure.sh
+
+RUN chmod +x /rits/runWithDockerEntrypoint.sh
+
+RUN tar xzf /rits/tcpsigner/bin/manager-tcp.tgz -C /rits/tcpsigner/bin
+
+# tcp signer dependencies
+RUN chmod +x /rits/tcpsigner/entrypoint.sh \
+    && chmod +x /rits/tcpsigner/bin/tcpsigner \
+    && chmod +x /rits/tcpsigner/bin/manager-tcp
+
+# Remove after debugging
+RUN apt-get update && apt-get install -y iproute2 && apt-get clean
+RUN apt update && apt install -y netcat-openbsd
+
+CMD [ "npm", "run", "test-fail-fast" ]

README.md

@@ -51,30 +51,29 @@ There are more variables in the `.env-example` files, but they already have defa
 
 `POWPEG_NODE_JAR_PATH` should point to the absolute path of the powpeg-node .jar file in your system to be executed.
 
-You can leave this value as the example or set `CONFIG_FILE_PATH` to the actual path of the configuration file you are going to use. If you don't provide a value, it will try to use `./config/regtest.js` instead.
+You can leave this value as the example or set `CONFIG_FILE_PATH` to the actual path of the configuration file you are going to use. If you don't provide a value, it will try to use `./config/regtest-all-keyfiles.js` instead.
 Notice that there is a `regtest-all-keyfiles.js` file in the `config` directory. You can either use that one, rename it, modify it or use a new one.
-At the moment, we are running the tests with `keyfiles` nodes. When we setup the tests to run with the `tcpsigner`, then we will update this guide to include that.
 
-Optional. If you want to see the logs of the federators (recommended), then `LOG_HOME` should point the directory where the logback configurations files are. Keep in mind that these logback `.xml` files should be contained in another directory inside this `LOG_HOME` directory called `genesis-federation` and `second-federation` and each of these directories should contain a `fed1.xml`, `fed2.xml` and `fed3.xml` files. You can always modify this in the `regtest-all-keyfiles.js` file.
+### Running the tests using the Tcp Signer
 
-Example:
+At the moment, we are running the tests with `keyfiles` nodes by default. If you want to run the tests with nodes using the tcp signer, then change the value of the `CONFIG_FILE_PATH` environment variable to `./config/regtest-key-files-and-hsms`. Using this configuration file will load one tcp signer instance for each federator that has an `hsm` type.
 
-```
-<LOG_HOME>
-  - genesis-federation/
-    - fed1.xml
-    - fed2.xml
-    - fed3.xml
-  - second-federation/
-    - fed1.xml
-    - fed2.xml
-    - fed3.xml
-```
+### Logs
+
+There is one `base-logback-config.xml` file in the `config` directory. It serves as an example of how the logback configuration files should look.
+
+In the `restest...` files in `config/` directory, each federator node has a reference to a logback configuration file path. By default, if the file doesn't exist in that path, then the setup process will create it automatically.
+
+You can update the `LOG_HOME` environment variable to point to where you have your logback configuration files. If you don't, then they will be created for you automatically at the root directory of this project, in a new `logs/` directory.
+
+The environment variable `LOG_HOME` should point the directory where the logback configurations files are.
 
-Use `container-action/rit-local-configs/logbacks` as an example.
+### Bitcoind
 
 You you already have `bitcoind` installed in your system, you can leave `BITCOIND_BIN_PATH` empty, the tests will use the one available from the system. If you have the `bitcoind` binaries in a specific directory, you can specify that directory in this variable. This way you don't have to fully install `bitcoind` in your system.
 
+### Java
+
 If you already have the correct `java` version installed in your system, you can leave `JAVA_BIN_PATH` empty, the tests will use the one available from the system. If you have the java binaries in a specific directory, you can specify that directory in this variable.
 
 Set the directory where you want the bitcoin database to be located at here `BITCOIN_DATA_DIR`.
@@ -211,3 +210,33 @@ The command `run-single-test-file` will execute the file `singleTestFileRunner.j
 1 - It will assign the `01_02_51-post_wasabi_fed_pubkeys_fork.js` test file name to the `process.env.INCLUDE_CASES` variable. Since it will be the one in that `INCLUDE_CASES` variable, then only that test file will be run.
 
 2 - It will setup a boolean `process.env.RUNNING_SINGLE_TEST_FILE` variable to `true` so the `fulfillRequirementsToRunAsSingleTestFile` function can check if it needs to manually take the blockchain to a state where the test file can run or not.
+
+### Running with docker
+
+You can simply run the `npm` command:
+
+> npm run run-with-docker
+
+Running it this way is important. It will read the `.env` variables first to locale the `POWPEG_NODE_JAR_PATH` and copy it here so `Dockerfile` can copy it along with the rest of the project.
+
+Using docker to run the tests you don't have to worry about installing the right version of Java or Bitcoind.
+
+It will also read the `DOCKER_REMOVE_CONTAINER_AFTER_EXECUTION` variable from the `.env` file.
+
+So make sure you have the `.env` file ready. Then, the `Dockerfile` will replace the `.env` file's content with the `.env.docker` file's content.
+
+#### Building the docker image and running the container directly
+
+Building
+
+To build the docker image directly, on the root directory, run:
+
+> docker buildx build --platform linux/amd64 -t rits .
+
+The `--platform linux/amd64` is necessary because the bitcoind binary depends on that platform, or it will fail to run.
+
+Running
+
+To run it, execute:
+
+> docker run --platform linux/amd64 -it rits

cleanEnv.js

@@ -55,7 +55,7 @@ function clearBitcoinDataDirectory() {
 const cleanEnvironment = async () => {
     console.info('Cleaning environment...');
     await clearBitcoinDataDirectory();
-    await clearLogFiles(process.env.LOG_HOME);
+    await clearLogFiles(process.env.LOG_HOME || './logs');
     shell.exec(killServicesCommand);
     console.info('Environment cleaned.');
 };