Skip to content

Latest commit

 

History

History
331 lines (234 loc) · 13.6 KB

user_guide.rst

File metadata and controls

331 lines (234 loc) · 13.6 KB
Raw

How to use Deckard

Contents

Deckard runs one or more binaries in isolated network which is described by so-called scenario. There are four components in play:

  • Deckard itself (test orchestrator)
  • binary under test (your own)
  • configuration for the binary (generated by Deckard from template, i.e. .j2 file)
  • environment description and test data (Deckard scenario, i.e. .rpl file)

It is easy to run tests if everything is already prepared and running tests gets harder as number of components you have to prepare yourself raises.

Let's start with the easiest case:

1   First run

Before we start, please note that Deckard depends on couple of modified C libraries. These will be automatically downloaded and compiled on first run, so do not be surprised when you see output from Git and C compiler:

$ make
Submodule 'contrib/libfaketime' (https://github.com/wolfcw/libfaketime.git) registered for path 'contrib/libfaketime'
Submodule 'contrib/libswrap' (https://gitlab.labs.nic.cz/labs/socket_wrapper.git) registered for path 'contrib/libswrap'
   [...]
-- The C compiler identification is GNU 6.3.1
   [...]
[ 50%] Building C object src/CMakeFiles/socket_wrapper.dir/socket_wrapper.c.o
   [...]
[100%] Built target socket_wrapper

For details see README.

2   Using an existing scenarios and configuration template

In the simplest case, the scenario and configuration template for given binary already exist and you just need to run them.

Most typically Deckard is not executed directly but through auxiliary scripts which supply parameters and run series of tests. For Knot Resolver, Unbound, and PowerDNS Recursor you can use scripts [run.sh] from Deckard repo. Following example shows script kresd_run.sh which executes tests on binary kresd (which must be in PATH).

$ ./kresd_run.sh
=== Testing WITH query minimization ===
   [...]
[100%] Built target socket_wrapper
[  OK  ] sets/resolver/iter_badglue.rpl
[  OK  ] sets/resolver/iter_cname_badauth.rpl
   [...]
=== Testing WITHOUT query minimization ===
   [...]
[  OK  ] sets/resolver/iter_badglue.rpl
[  OK  ] sets/resolver/iter_cname_badauth.rpl
   [...]

Return code will be 0 if all tests passed, non-zero otherwise.

[run.sh](1, 2, 3, 4, 5) See scripts kresd_run.sh, unbound_run.sh, and pdns_run.sh in Deckard repo.

2.1   Tweaking script execution

Behavior of *_run.sh scripts can be modified using following environment variables:

  • DAEMON - path to binary under test
  • ADDITIONAL - additional parameters for binary under test (e.g. path to configuration files)
  • CONFIG - colon-separated list of names of configuration files to be generated from Jinja2 templates
    • CONFIG files will be generated from respective files in TEMPLATE list
    • i.e. the first file in CONFIG list is the result of processing of the first file from TEMPLATE list and so on
    • generated files are stored in a new working directory created by Deckard for each binary
  • TEMPLATE - colon-separated list of Jinja2 template files to generate configuration files
  • TESTS - path to scenario file or directory with scenario files to be recursively processed
  • MAKEFLAGS - scripts internally use make so all relevant options can be used
    • feel free to set environment variable MAKEFLAGS to run tests in parallel, e.g. export MAKEFLAGS="-j4"

Commented default values taken from unbound_run.sh follow:

# Path to daemon
DAEMON="unbound"

# Additional parametes for binary: configuration file is in working directory
ADDITIONAL="-d -c unbound.conf"

# Template file names
TEMPLATE="template/unbound.j2:template/hints_zone.j2"

# Config file names: generated respectively from templates above
CONFIG="unbound.conf:hints.zone:ta.keys"

# Run all tests in directory "sets/resolver"
TESTS="sets/resolver"

Most often it is sufficient to use these variables for basic configuration changes. Read next section for details about config file templates.

3   Using an existing scenarios with custom configuration template

It some cases it is necessary to modify or create new template files. Typically this is needed when:

  • there are no templates for particular binary (e.g. if you want to test a brand new program)
  • an existing template hardcodes some configuration and you want to change it

Deckard uses the Jinja2 templating engine (like Ansible or Salt) and supplies several variables that you can use in templates. For simplicity you can imagine that all occurrences of {{variable}} in template are replaced with value of the variable. See Jinja2 documentation for further details.

Here is an example of template for Unbound:

server:
     directory: ""                 # do not leave current working directory
     chroot: ""
     pidfile: ""
     username: ""

     interface: {{SELF_ADDR}}      # Deckard will assign an address
     interface-automatic: no
     access-control: ::0/0 allow   # accept queries from Deckard

     do-daemonize: no              # log to stdout & stderr
     use-syslog: no
     verbosity: 3                  # be verbose, it is handy for debugging
     val-log-level: 2
     log-queries: yes

     {% if QMIN == "false" %}      # Jinja2 condition
     qname-minimisation: no        # a constant inside condition
     {% else %}
     qname-minimisation: yes
     {% endif %}
     harden-glue: no               # hardcoded constant, use a variable instead!

     root-hints: "hints.zone"      # reference to other files in working directory
     trust-anchor-file: "ta.keys"  # use separate template to generate these

This configuration snippet refers to files hints.zone and ta.keys which need to be generated as well. Each file uses own template file. An template for hints.zone might look like this:

# this is hints file which directs resolver to query
# fake root server simulated by Deckard
.                        3600000      NS    K.ROOT-SERVERS.NET.
# IP address version depends on scenario setting, handle IPv4 & IPv6
{% if ':' in ROOT_ADDR %}
K.ROOT-SERVERS.NET.      3600000      AAAA  {{ROOT_ADDR}}
{% else %}
K.ROOT-SERVERS.NET.      3600000      A     {{ROOT_ADDR}}
{% endif %}

Templates can use any of following variables:

3.1   List of variables for templates

Addresses:

  • DAEMON_NAME - user-specified symbolic name of particular binary under test, e.g. recursor
  • IPADDRS - dictionary with {symbolic name: IP address} mapping
    • it is handy for cases where configuration for one binary under test has to refer to another binary under test
  • ROOT_ADDR - fake root server hint (Deckard is listening here; port is not expressed, must be 53)
    • IP version depends on settings in particular scenario
    • templates must handle IPv4 and IPv6 as well
  • SELF_ADDR - address assigned to the binary under test (port is not expressed, must be 53)

Path variables:

  • INSTALL_DIR - path to directory containing file deckard.py
  • WORKING_DIR - working directory for binary under test, each binary gets its own directory

DNS specifics:

  • DO_NOT_QUERY_LOCALHOST [bool] - allows or disallows querying local addresses
  • HARDEN_GLUE [bool] - enables or disables additional checks on glue addresses
  • QMIN [bool] - enables or disables query minimization respectively
  • TRUST_ANCHORS - list of trust anchors in form of a DS records, see scenario guide
[bool](1, 2, 3) boolean expressed as string true/false

It's okay if you don't use all of the variables, but expect some tests to fail. E.g. if you don't set the TRUST_ANCHORS, then the DNSSEC tests will not work properly.

3.2   Running custom templates

Custom templates can be used in the same way as templates listed in existing [run.sh] scripts. During template development it might be handy to use make variables for quick prototyping:

make \
    TESTS="sets/resolver"                                                            \
    DAEMON="unbound"                                                                 \
    ADDITIONAL="-d -c unbound.conf"                                                  \
    TEMPLATE="template/unbound.j2:template/hints_zone.j2"                            \
    CONFIG="unbound.conf:hints.zone:ta.keys"

(These are the default values for Unbound.)

4   Debugging scenario execution

Output from a failed test looks like this:

$ ./kresd_run.sh
   [...]
[ FAIL ] sets/resolver/iter_cname_cache.rpl
sets/resolver/iter_cname_cache.rpl step 50 line 283, "rcode": expected 'NOERROR', got 'SERVFAIL' in the response:
id 12540
opcode QUERY
rcode SERVFAIL
flags QR RD
edns 0
payload 4096
;QUESTION
ns.bla.nl. IN AAAA
;ANSWER
;AUTHORITY
;ADDITIONAL
Traceback (most recent call last):
  File "/home/pspacek/pkg/deckard/git/pydnstest/test.py", line 25, in run
    test_callback(name, args, config)
  File "/home/pspacek/pkg/deckard/git/deckard.py", line 290, in play_object
    server.play(prog_under_test_ip)
  File "/home/pspacek/pkg/deckard/git/pydnstest/testserver.py", line 198, in play
    self.scenario.play({'': (subject_addr, 53)})
  File "/home/pspacek/pkg/deckard/git/pydnstest/scenario.py", line 788, in play
    raise Exception('%s step %d %s' % (self.file, step.id, str(e)))

In this example, the test step 50 in scenario sets/resolver/iter_cname_cache.rpl is failing. The binary under test did not produce expected answer, so either the test scenario or binary is wrong. If we were debugging this example, we would have to open file iter_cname_cache.rpl on line 283 and use our brains :-)

Tips:

  • details about scenario format are in the scenario guide
  • network traffic from each binary is logged in PCAP format to a file in working directory
  • standard output and error from each binary is logged into log file in working directory
  • working directory can be explicitly specified in environment variable SOCKET_WRAPPER_DIR
  • environment variable VERBOSE=1 forces extra verbose logging, including logs from all binaries and packets handled by Deckard

5   Writting own scenarios

See the scenario guide.

6   Direct Deckard invocation

Warning

Direct Deckard invocation is typically used only for development. The command line interface is not stable!

Usually Deckard is invoked using make or even higher-level scripts like [run.sh]. The main reason is that each Deckard invocation requires LD_PRELOAD variable set to custom versions of C libraries which are used for environment simulation.

If you really have to, you can run Deckard directly. Single scenario can be executed in two distinct modes named one and multiple:

  • one - read all parameters from command-line and run one binary
  • multiple - read parameters from YAML file and run multiple binaries

For all the details please see built-in help, run deckard.py --help to see it.

6.1   One binary mode (one)

This mode runs one binary with parameters specified on command line and is the most common usage of Deckard. It is used by supplied [run.sh] scripts (indirectly through make). Use MAKEFLAGS to see what the [run.sh] script is executing:

$ MAKEFLAGS="--dry-run" ./kresd_run.sh
LD_PRELOAD=".../contrib/..." deckard.py sets/resolver/world_mx_nic_www.rpl one kresd template/kresd.j2 config --

6.2   Multiple binaries mode (multiple)

In this mode Deckard reads YAML configuration file and executes all binaries using parameters from the file. This is handy for testing interoperability of multiple binaries, e.g. when one program is configured as DNS recursor and other program is using it as forwarder.

The YAML file contains ordered list of binaries and their parameters. Deckard will send queries to the binary listed first.

programs:
- name: forwarding            # name of this Knot Resolver instance
  binary: kresd               # kresd is first so it will receive queries from Deckard
  additional: []
  templates:
    - template/kresd_fwd.j2   # this template uses variable IPADDRS['recursor']
  configs:
    - config
- name: recursor              # name of this Unbound instance
  binary: unbound
  additional:
    - -d
    - -c
    - unbound.conf
  templates:
    - template/unbound.j2
    - template/hints_zone.j2  # this template uses variable ROOT_ADDR
  configs:
    - unbound.conf
    - hints.zone
    - ta.keys

In this setup it is necessary to configure one binary to contact the other. IP addresses assigned by Deckard at run-time are accessible using IPADDRS template variables and symbolic names assigned to binaries in the YAML file. For example, template kresd_fwd.j2 can use IP address of binary named recursor like this:

policy.add(policy.all(policy.FORWARD("{{IPADDRS['recursor']}}")))

When all preparations are finished, run Deckard using following syntax:

$ LD_PRELOAD=".../contrib/..." deckard.py --loglevel INFO scenario.rpl multiple config.yml