A tool to automate connectivity checks.
Create one or more bundles, each containing the set of checks to run.
It's possible to write bundles in JSON or YAML format. See directory _tests for examples.
Supported protocols include TCP, UDP, ICMP, SSH, HTTP, HTTPs, TLS over streams (TLS) and TLS over datagrams (DTLS), the latter three including certificate verification; TCP, UDP, SSH, TLS and DTLS checks require an address including hostname/IP address and port (host.example.com:80 or 192.168.1.15:443); ICMP checks only require the hostname or IP address; HTTP, HTTPS and SSH checks will use the default protocol ports (80, 443 and 22 respectively) if none is specified.
HTTP/HTTPS and SSH checks behave differently from TCP and TLS checks in that they also try to establish a valid connection at the application level of the network stack. This means that while a TCP check to example.com:80 will connect to the web server example.com on port 80 and report whether the packet flow succeeded, an HTTP test against the same address will also check if the response is a valid (HTTP 200 status code) response; the HTTPS check (typically run against port 443) will additionally perform a TLS handshake and check that the server certificate is valid. When the sso flag is set to true, the check will also try to use the local Kerberos/NTLM identity to authenticate the request on the web server.
It's possible to specify the default timeout for the whole bundle, or more specific timeouts for each check within a bundle. Moreover it's possible to specify how many times to retry in case of failure and a wait time between attempts.
This is a sample bundle in YAML format:
id: my-bundle
description: a collection of useful checks
timeout: 5s # this applies by default to all checks
concurrency: 10 # run these many checks concurrently
retries: 3 # in case of failure, try these many times...
wait: 5s # ... waiting this long between attempts
checks:
- address: www.google.com:80 # hostname:port
protocol: tcp # TCP is the default: it can be omitted (see below)
timeout: 1s # specify a different timeout
- address: www.google.com:443 # hostname:port, all the rest is the default
- address: dns.example.com:53
protocol: udp # use UDP for DNS
- address: www.google.com # ping this host
protocol: icmp
- address: github.com:22 # try to SSH to this host
protocol: ssh
- name: Google (HTTPs with page) # ... with its own check name
address: www.google.com/imghp?hl=en&authuser=0&ogbl
protocol: https # this is HTTPs, you can test a specific resource!
sso: trueBundles can be:
- local (i.e. a file on disk)
- remotely GET-table HTTP/HTTPs resources (i.e., specified as an HTTP URL),
- values in Consul key/value stores (i.e. stored as a Consul value and pointed at through its key)
- values in Consul Service Registry services' metadata (i.e. stored in the service's metadata)
- values in Redis key/value stores (i.e. stored ina Redis value and pointed at through the key)
These things can be mixed, so you can call netcheck on multiple bundles at once, mixing them at will.
All checks will be performed bundle by bundle, in the same order that was specified on the command line.
The application supports retrieving bundles from authenticated HTTP/HTTPS server. To use basic authentication, use the ordinary syntax (e.g. https-://<username>:<password>@example.com:443); to retrieve it from a Kerberos/NTLM protected server, you have to specify a custom schema with an additional +sso( e.g. https+sso-://example.com/ for HTTPS with single sign on and without certificate).
The output can be in text mode (the default), in one of json and yaml formats, or generated dynamically in an arbitrary format based on a user-provided Golang template.
$> netcheck --format=json local-1.yaml local-2.json \
http://remote.example.com?id=1 \
https://remote.example.com/remote-2.jsonWhen redirected to file, the text mode is not colorised.
When exposing remote bundles via HTTP, make sure the Content-Type is properly set, as it is used to identify the format of the checks bundle (YAML, JSON).
The following is an example output of running the check against a local bundle:
$> netcheck --format=yaml test.yaml
test-bundle:
- protocol: tcp
endpoint: www.google.com:443
success: true
- protocol: tcp
endpoint: www.repubblica.it:443
success: true
- protocol: tcp
endpoint: www.repubblica.it:443
success: falseThe application assumes that any argument that is not recognised as a command line parameter is the URL of a bundle.
Bundles can be retrieved from multiple sources: a local file, an HTTP server, a Consul Key/Value store, a Consul Service Registry, a Redis instance. If the argument is not parsed as a valid URL, it is assumed to point to a local file.
The application supports downloading a bundle from an HTTP or HTTPs server. The URL is usually an ordinary HTTP address, with the exception that in order to skip the TLS certificate verification, the https-:// custom scheme is supported. The - is the same as specifying -k with cURL or --insecure-skip-verify on many other applications.
The application requires a custom schema to identify HTTP/HTTPS resources that require Kerberos/NTLM authetication:
http+sso://for HTTP with Kerberos/NTLM authentication,https+sso://for HTTPS with Kerberos/NTLM authentication, andhttps+sso-://for HTTPS with Kerberos/NTLM authentication and without certificate verification.
The application supports downloading a bundle from a Redis server, in plaintext or with a TLS-protected protocol. The URL is prefixed with the redis:// scheme for plaintext, rediss:// for secure-Redis, and rediss-:// for secure-Redis with skipped verification of the TLS certificate. The URL must also contain the key query parameter to specify the key under which the bundle is stored, and can optionally have the db query parameter if the key is on a non-default (!= 0) database.
The application supports downloading a bundle from a Consul Key/Value store, in plaintext or with a TLS-protected protocol. The URL is prefixed with the consulkv:// scheme for plaintext, consulkvs:// for secure-Consul, and consulkvs-:// for secure-Consul with skipped verification of the TLS certificate. The URL must also contain the key query parameter to specify the key under which the bundle is stored, and can optionally have the dc query parameter if the key is not in the default datacentre.
The application supports downloading a bundle from a Consul Service Registry, in plaintext or with a TLS-protected protocol. The URL is prefixed with the consulsr:// scheme for plaintext, consulsrs:// for secure-Consul, and consulsrs-:// for secure-Consul with skipped verification of the TLS certificate. The URL must also contain the service query parameter to specify the name of the service, an optional tag to help filtering on the list of services in the registry, and a compulsory meta value which represents the name of the service metadata map under which the bundle is stored in JSON or YAML format; moreover it can optionally have the dc query parameter if the service is not in the default datacentre.
When the --template=<mytemplate.tpl> command line parameter is specified, it overrides the --format parameter setting it to template; the application will then proceed to compile the provided template and use it on the following data structure:
[]struct {
ID string // the id of the bundle
Description string // a description of the bundle
Timeout Timeout // the connection timeout
Retries int // how many attempts before declaring failure...
Wait Timeout // and how long to wait between those successive attempts
Concurrency int // how many checks to run concurrently
Checks []struct {
Description string // the description of the check
Timeout Timeout // the connection timeout (to override the bundle-global one)
Retries int // how many attempts before declaring failure...
Wait Timeout // and how long to wait between those successive attempts
Address string // the address to connect to, possibly including the port
Protocol int // to translate this to "icmp", "tls"... use the .String method
SSO bool // whether to use single-sign-on with SPNEGO authentication
Result Result // the check's result, see below for details
} // the array of checks in the bundle
}The Result structure (inside each of the Checks in the Bundle) provides two utility methods:
String(), which either returns the string"success"or the string representation of the error, andIsError()that provides a way to check if the result represents a failure.
They can be used in the output template too, as shown in the _tests/output.tpl file, which provides an extensive example:
{{ range . }}
Bundle : {{ .ID | cyan }}{{ if .Description }}
Description : {{ .Description | cyan }}{{ end }}
Timeout : {{ .Timeout.String | cyan }}
Retries : {{ .Retries | cyan }} attempts before failing
Wait time : {{ .Wait | cyan }} between successive attempts
Concurrency : {{ .Concurrency | cyan }} concurrent goroutines
--------------------------------------------------------------------------------{{ range .Checks }}
- Protocol: {{ .Protocol.String | purple }}
Host: {{ $a := splitList ":" .Address }}{{ index $a 0 | yellow }}{{ $l := len $a }}{{ if eq $l 2 }}
Port: {{ index $a 1 | yellow }}{{ end }}
{{ if .Result.IsError }}Result: {{ .Result.String | red }}{{ else }}Result: {{ .Result.String | green }}{{ end }}{{ end }}
{{ end }}--------------------------------------------------------------------------------The first range loop goes over the array of bundles, Bundle by bundle; . will refer to the current bundle; some information about the bundle is printed out.
The second range loop runs over the array of Checks within the bundle and prints out:
- the protocol: see the use of
.Protocol.Stringto print the textual representation of the protocol, - the host: see how the
splitListSprig function is used to split hostname/IP and port apart - the port: only if the
splitListoperation returned more than one item (ICMP does not have a port!) - the error: only if it is not nil
Note: The template engine includes the excellent Sprig library functions to help with values manipulation, plus some additional colouring functions (blue, cyan, green, magenta, purple, red, yellow, white and their "highlighted" version: hiblue, hicyan...); the usage is shown in the _test/output.tpl template and in the previous example.
If you want to try your template without having to wait for real checks to be performed, call the application with the --template=<mytemplate.tpl> parameter and no bundle: the application will generate a mock result that includes a couple of bundles with two checks each, and will use it to apply the provided template.
Moreover, if you pass the --print-diagnostics flag, the application will also print out a representation of the mock result where the fields that were accessed by the template are highlighted in magenta. This can help you understand the data structure.
$> netcheck --template=_test/output.tpl --print-diagnosticsThe application can optionally load the default values to use when not provided in the bundle.
On Unix systems it tries to load the defaults from a file called netcheck.conf, in YAML format, in the following paths: ./netcheck.conf, ~/netcheck.conf, /etc/netcheck.conf.
On Windows systems it tries to load the defaults from a file called netcheck.conf, in YAML format, in the following paths: ./netcheck.conf, ~/netcheck.conf.
The file containing the default values, that is installed alongsite the application under /etc, is in the sources root directory:
timeout: 2s # fail a check after 2 seconds without response
retries: 3 # try up to 3 times if check fails
wait: 100ms # wait 100 millisendons between attempts
concurrency: 10 # run 10 checks concurrently
ping:
count: 10 # send 10 packets
interval: 100ms # send an ICMP packet every 100 microseconds
size: 64 # 64 bytesThe application is pre-built for a multiplicity of platforms (Linux, Windows, Mac) and architectures (AMD64, ARM64), thanks to Golang support for a lot of architectures. Moreover, thanks to nFPM, it comes packaged in many installable formats including DEB, RPM and APK. To download the binary file, go to the project's GitHub page at https://github.com/dihedron/netcheck and then refer to your OS package manager for installation instructions.
Compilation requires Golang 1.25+, make and goreleaser.
In order to build, run make compile. Running make help provides list of all the choices. By default make and the default target make compile build for linux/amd64.
In order to release a new version, commit all outstanding changes, then create a tag with the new version:
$> git tag -a v1.2.3 -m "v1.2.3 - bug fixes and updated dependencies"
$> git push origin v1.2.3There is no need to run make release as the GitHub action will do it for you.
To run HTTPs unit tests, run make self-signed-cert to generate the fetch/server.key and fetch/server.crt that will be used by the local HTTPs server.
Run under the NETCHECK_LOG_LEVEL=debug environment variable; other acceptable log levels are info, warn, error and off (the default).
- Evaluate whether/how to allow loading of custom trust anchors for TLS validation.
- Implement support for single-sign-on with SPNEGO authentication in HTTP(s) bundle retrieval