Dockerfile for backend

[MichaelTimbert]

Purpose

This PR propose a Dockerfile for backend.
It capable to launch 'rpcapi' or 'testagent' or both from docker run argument.
'zmb' and 'zmtest' are also available from docker run argument.
It is also capable to launch zonemaster-cli, it is needed for building an All In On docker image able to launch both CLI or GUI.
This docker image use S6 Overlay to manage multiple process running at the same time in docker.

It's possible to launch only 'rpcapi' or 'testagent' alone. It's for future use, and is not useful at the moment.

Usage

For testing zonemaster-cli

$ docker run -ti --rm zonemaster/backend:local cli test.fr

All argument after 'cli' are forwarded to 'zonemaster-cli'

For testing zonemaster backend

$ docker run --rm -p 5000:5000 --name zm -d zonemaster/backend:local full

Container can be stopped with docker stop zm

You can interact with the backend with 'zmb' or 'zmtest' from the host if they are installed. Otherwise, you can use them from within Docker itself.

docker run -ti --rm --net host zonemaster/backend:local zmtest test.fr
testid: ed38765834e45b6e
1% done

docker run -ti --rm --net host zonemaster/backend:local zmb start_domain_test --domain test.fr
{"id":1,"result":"ed38765834e45b6e","jsonrpc":"2.0"}
docker run -ti --rm --net host zonemaster/backend:local zmb get_test_results --test-id ed38765834e45b6e --lang en

Note: The "--net host" option is required, otherwise the container cannot access the backend on port 5000.

Context

Proposition for #953

Changes

• Change Makefile.PL to add 'docker-build' target
• Add a Dockerfile based on zonemaster-cli image
• Add 'zonemaster_launch' script for the docker entrypoint

How to test this PR

To be able to test this PR, you need to build zonemaster-cli docker image first.
You must be able to build 'zonemaster/backend:local' image by executing

perl Makefile.PL
make all dist docker-build

Testing zonemaster-cli

You must be able to test zonemaster-cli with docker run -ti --rm zonemaster/backend:local cli [options]. Every option must be passed to zonemaster-cli without exception.

Testing backend

launch the backend with the following cmd:

$ docker run --rm -p 5000:5000 --name zm -d zonemaster/backend:local full

You must be able to test some domain using zmb from the host.
To stop the backend run:

docker stop zm

[matsduf]

With this we have two docker images with zonemaster-cli (which is fine). Since this one inherits the other can it then be assumed that they two will have the same behavior?

[matsduf]

You must be able to test some domain using zmb from the host.

Can zmb be run with

$ docker run -ti --rm zonemaster/backend:local zmb test.fr

[MichaelTimbert]

With this we have two docker images with zonemaster-cli (which is fine). Since this one inherits the other can it then be assumed that they two will have the same behavior?

not exactly, we need to add cli argument before any zonemaster-cli arguments:

docker run -ti --rm zonemaster/cli [zonemaster-cli options] test.fr
vs
docker run -ti --rm zonemaster/backend:local cli [zonemaster-cli options] test.fr

I don't think this is a problem if the end goal is to have one image to run all Zonemaster programs, like:

docker run -ti --rm zonemaster cli ...
docker run -ti --rm zonemaster zmtest ...
docker run -ti --rm zonemaster full ...

[MichaelTimbert]

You must be able to test some domain using zmb from the host.

Can zmb be run with

$ docker run -ti --rm zonemaster/backend:local zmb test.fr

We can do it by reusing the same container :

docker run -ti --rm --name zm  zonemaster/backend:local full
docker exec zm zmb start_domain_test --domain test.fr
docker exec zm zmb get_test_results --test-id 8cea2bea98449ea9 --lang en

or using another container, but in this case the container must use the --net host option to work.

docker run -ti --rm --name zm -p 5000:5000 zonemaster/backend:local full
docker run -ti --rm --net host --entrypoint zmb zonemaster/backend:local start_domain_test --domain test.fr
docker run -ti --rm --net host --entrypoint zmb zonemaster/backend:local get_test_results --test-id b1dac7ceb8b95f33 --lang en

If we want, we can add "zmb" and "zmtest" to the zonemaster_launch script to be able to run it, like this:

docker run -ti --rm --name zm -p 5000:5000 zonemaster/backend:local full
docker run -ti --rm --net host zonemaster/backend:local zmb start_domain_test --domain test.fr
docker run -ti --rm --net host zonemaster/backend:local zmtest test.fr

[matsduf]

With this we have two docker images with zonemaster-cli (which is fine). Since this one inherits the other can it then be assumed that they two will have the same behavior?

not exactly, we need to add cli argument before any zonemaster-cli arguments:

Yes, but that is close enough to me. I should have clarified that in my question.

[matsduf]

I don't think this is a problem if the end goal is to have one image to run all Zonemaster programs, like:

I think that we should still keep the zonemaster-cli docker image for those that just want to run that. The extra argument for CLI when using the "full" image is not a problem, I think.

[matsduf]

I think the following alternative looks fine. It is just a matter of documentation.

If we want, we can add "zmb" and "zmtest" to the zonemaster_launch script to be able to run it, like this:

docker run -ti --rm --name zm -p 5000:5000 zonemaster/backend:local full
docker run -ti --rm --net host zonemaster/backend:local zmb start_domain_test --domain test.fr
docker run -ti --rm --net host zonemaster/backend:local zmtest test.fr

[matsduf]

For testing zonemaster backend

$ docker run --rm -p 5000:5000 --name zm -d zonemaster/backend:local full

As I understand it, you can choose another ports, but what do the two "5000" mean? Some port translation?

docker run -ti --rm --net host zonemaster/backend:local zmb start_domain_test --domain test.fr
{"id":1,"result":"ed38765834e45b6e","jsonrpc":"2.0"}
docker run -ti --rm --net host zonemaster/backend:local zmb get_test_results --test-id ed38765834e45b6e --lang en

Note: The "--net host" option is required, otherwise the container cannot access the backend on port 5000.

If you choose another port for RPCAPI, e.g. 6000, can you specify that to zmb with --server http://localhost:6000/?

[matsduf]

To be able to test this PR, you need to build zonemaster-cli docker image first. You must be able to build 'zonemaster/backend:local' image by executing make docker-build

Just running make docker-build did not work. But the following seems to work:

perl Makefile.PL
make all dist docker-build

[matsduf]

(ipv6 is most likely to not work inside docker).

I had no issues with IPv6. It works for both CLI and Backend via zmtest/zmb. I tested on a Ubuntu VPS where I have enabled IPv6 support for Docker. See instructions in https://github.com/zonemaster/zonemaster/blob/569501e0afa34de739ce3fb7c2ccd1a7d6f39161/docs/internal/maintenance/ReleaseProcess-create-docker-image.md#enable-ipv6-support

[matsduf]

I had an issue with piping the output of an zmb command through jq. The zmb command outputted a trailing ^M ("CR" code) that messed up the output of jq. I found a work-around by ping through a tr command, e.g.

docker run -ti --rm --net host zonemaster/backend:local zmb get_test_results (...) |tr -d '^M' | jq

Would it be possible to have the ^M removed already in the Docker container?

[matsduf]

There are some rough edges, but this can be merged as far as I can see. The "how to test should be updated" too.

I assume that documentation is added in various documents in documentation tree.

[MichaelTimbert]

For testing zonemaster backend

$ docker run --rm -p 5000:5000 --name zm -d zonemaster/backend:local full

As I understand it, you can choose another ports, but what do the two "5000" mean? Some port translation?

docker run -ti --rm --net host zonemaster/backend:local zmb start_domain_test --domain test.fr
{"id":1,"result":"ed38765834e45b6e","jsonrpc":"2.0"}
docker run -ti --rm --net host zonemaster/backend:local zmb get_test_results --test-id ed38765834e45b6e --lang en

Note: The "--net host" option is required, otherwise the container cannot access the backend on port 5000.

If you choose another port for RPCAPI, e.g. 6000, can you specify that to zmb with --server http://localhost:6000/?

Yes -p 5000:5000 mean some port translation: port 5000 of localhost is forwarded to port 5000 of the container.

You can change the port, for example -p 2304:5000 and use --server http://localhost:2304 option on zmb.

[MichaelTimbert]

To be able to test this PR, you need to build zonemaster-cli docker image first. You must be able to build 'zonemaster/backend:local' image by executing make docker-build

Just running make docker-build did not work. But the following seems to work:

perl Makefile.PL
make all dist docker-build

Thanks, I've edited it.

[MichaelTimbert]

I had an issue with piping the output of an zmb command through jq. The zmb command outputted a trailing ^M ("CR" code) that messed up the output of jq. I found a work-around by ping through a tr command, e.g.

docker run -ti --rm --net host zonemaster/backend:local zmb get_test_results (...) |tr -d '^M' | jq

Would it be possible to have the ^M removed already in the Docker container?

Thanks for the tips, I can't find a way to remove it directly from the docker output, I'll keep looking for it.

[matsduf]

Would it be possible to have the ^M removed already in the Docker container?

Thanks for the tips, I can't find a way to remove it directly from the docker output, I'll keep looking for it.

If it cannot be removed we should add that to the documentation. It might not be needed under Windows.

[matsduf]

Will we get some documentation under https://github.com/zonemaster/zonemaster/tree/master/docs/public/using/backend? It would be good with some cross references between that new document and https://github.com/zonemaster/zonemaster/blob/master/docs/public/using/cli.md

[marc-vanderwal]

Successfully tested this PR for release 2025.2.