Merge branch 'main' into feature/456

CONTRIBUTING.md

@@ -74,6 +74,33 @@ After the correct controller-gen version is installed.  All future builds will w
 
 After running this command, CLI will be available in `bin/kubectl-kuttl` and you can invoke the command for example like this `bin/kubectl-kuttl version` (no need to install it as `kubectl` plugin).
 
+### Debugging in VSCode
+
+Click Run > Add Configuration... to open the `launch.json` file. Add below configuration objects to specify the necessary sample tests and assertions in the `args[]`.
+
+```json
+{
+  // Use IntelliSense to learn about possible attributes.
+  // Hover to view descriptions of existing attributes.
+  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "KUTTL Debug",
+      "type": "go",
+      "request": "launch",
+      "mode": "debug",
+      "program": "${workspaceFolder}/cmd/kubectl-kuttl/main.go",
+      "args": [
+        "test", 
+        "sample", 
+        "--config=${workspaceFolder}/cmd/kubectl-kuttl/<sample_tests_dir_to_run_with_kuttl>"
+      ]
+    }
+  ]
+}
+```
+
 ### Testing
 
 Use `make all` to run all available tests.

Dockerfile

@@ -26,6 +26,7 @@ RUN make cli
 
 # release image with kubectl + kuttl
 FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
+LABEL org.opencontainers.image.source="https://github.com/kudobuilder/kuttl"
 
 RUN microdnf install vim tar gzip
 RUN echo 'alias vi=vim' >> ~/.bashrc

README.md

@@ -5,15 +5,15 @@
 [![lint](https://github.com/kudobuilder/kuttl/actions/workflows/lint.yml/badge.svg?branch=main)](https://github.com/kudobuilder/kuttl/actions)
 [![unit test](https://github.com/kudobuilder/kuttl/actions/workflows/unittest.yml/badge.svg?branch=main)](https://github.com/kudobuilder/kuttl/actions)
 [![integration-test](https://github.com/kudobuilder/kuttl/actions/workflows/integration-test.yml/badge.svg?branch=main)](https://github.com/kudobuilder/kuttl/actions)
-[![e2e](https://github.com/kudobuilder/kuttl/actions/workflows/e2e.yml/badge.svg?branch=main)](https://github.com/kudobuilder/kuttl/actions)
+[![Go Report Card](https://goreportcard.com/badge/github.com/kudobuilder/kuttl)](https://goreportcard.com/report/github.com/kudobuilder/kuttl)  
 
 KUbernetes Test TooL (KUTTL) provides a declarative approach to test Kubernetes Operators.
 
 KUTTL is designed for testing operators, however it can declaratively test any kubernetes objects.
 
 ## Getting Started
 
-Please refer to the [getting started guide](https://kuttl.dev/docs/) documentation.
+Please refer to the [getting started guide](docs/README.md) documentation.
 
 ## Resources
 
@@ -36,7 +36,7 @@ Learn more on how to engage with the KUDO community on the [community page](http
 
 ## Contributions
 
-Please read the [contributing guide](https://github.com/kudobuilder/kuttl/blob/main/CONTRIBUTING.md) for details around:
+Please read the [contributing guide](CONTRIBUTING.md) for details around:
 
 1. Code of Conduct
 1. Code Culture

docs/README.md

@@ -0,0 +1,24 @@
+# Getting Started
+
+## Pre-requisites
+
+Before you get started using KUTTL, you need to have a running Kubernetes cluster setup. If you already have a cluster there are no prerequisites.  If you want to use the mocked control plane or Kind, you will need [Kind](https://github.com/kubernetes-sigs/kind).
+
+- Setup a Kubernetes Cluster in version `1.13` or later
+- Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) in version `1.13` or later.
+
+## Install KUTTL CLI
+
+Install the `kubectl kuttl` plugin. To do so, please follow the [CLI plugin installation instructions](cli.md).
+
+The KUTTL CLI leverages the kubectl plugin system, which gives you all its functionality under `kubectl kuttl`.
+
+## Using KUTTL
+
+Once you have a running cluster with `kubectl` installed along with the KUTTL CLI plugin, you can run tests with KUTTL like so:
+
+```bash
+$ kubectl kuttl test path/to/test-suite
+```
+
+[Learn more](what-is-kuttl.md) about KUTTL and check out how to get started with the [KUTTL test harness](kuttl-test-harness.md).

docs/api-integration.md

@@ -0,0 +1,53 @@
+# API Integration
+
+It is possible to integrate KUTTL into your own Go test infrastructure.  KUDO provides as an example `kubectl kudo test` using the KUTTL test harness.  The following are the necessary steps.
+
+## Add KUTTL to Go.mod
+
+`go get github.com/kudobuilder/kuttl`
+
+or get a specific version
+
+`go get github.com/kudobuilder/[email protected]`
+
+## Common Imports to Use
+
+The test harness type is defined in an `apis` package similar to a Kubernetes type along with a version package.  The test harness is currently `v1beta1` and provides the main configuration for a test suite.
+
+The `test` package contains the `test.Harness` implementation (given the configuration of the test harness configuration type previously mentioned).  The `test.Harness` provides the "run" of the test run and needs a Go `t *testing.T`.
+
+The `testutils` package contains utilities for docker, kubernetes, loggers and testing.
+
+```go
+import (
+  harness "github.com/kudobuilder/kuttl/pkg/apis/testharness/v1beta1"
+  "github.com/kudobuilder/kuttl/pkg/test"
+  testutils "github.com/kudobuilder/kuttl/pkg/test/utils"
+)
+```
+
+## Test Harness
+
+The `harness.TestSuite` is the structure that controls how the test harness will run.
+
+```go
+options := harness.TestSuite{}
+```
+
+The Go `t *testing.T` and `harness.TestSuite` are provided to `test.Harness` which provides the implementation for testing.
+
+```go
+Run: func(cmd *cobra.Command, args []string) {
+  testutils.RunTests("kudo", testToRun, options.Parallel, func(t *testing.T) {
+    harness := test.Harness{
+      TestSuite: options,
+      T:         t,
+    }
+
+    harness.Run()
+  })
+},
+
+```
+
+A more complete example is provided in KUDOs [cmd/test.go](https://github.com/kudobuilder/kudo/blob/master/pkg/kudoctl/cmd/test.go)

docs/cli.md

@@ -0,0 +1,151 @@
+# CLI Usage
+
+This document demonstrates how to use the KUTTL CLI
+
+## Setup the KUTTL Kubectl Plugin
+
+### Requirements
+
+- `kubectl` version `1.13.0` or newer
+
+### Installation
+
+You can either download CLI binaries for linux or MacOS from our [release page](https://github.com/kudobuilder/kuttl/releases), or install the CLI plugin using `brew`:
+
+```bash
+brew tap kudobuilder/tap
+brew install kuttl-cli
+```
+
+or you can compile and install the plugin from your `$GOPATH/src/github.com/kudobuilder/kuttl` root folder via:
+
+```bash
+make cli-install
+```
+
+Another alternative is [`krew`](https://github.com/kubernetes-sigs/krew), the package manager for kubectl plugins.
+
+```bash
+kubectl krew install kuttl
+```
+
+## Commands
+
+* **`kubectl kuttl help [command] [flags]`**
+
+  Provide general help or help on a specific command
+
+* **`kubectl kuttl version`**
+
+  Print the current KUTTL version.
+
+* **`kubectl kuttl test`**
+
+  Run KUTTL test harness.
+
+
+## Flags
+
+> [!NOTE]
+> **Usage**
+>
+> `kubectl kuttl test <name> [flags]`
+
+Flags are:
+
+* **`-h, --help`**
+
+  Help for test
+
+* **`--artifacts-dir (string)`**
+
+  Directory to output kind logs to (if not specified, the current working directory).
+
+* **`--config (string)`**
+
+  Path to file to load test settings from. This is usually the `kuttl-test.yaml` file.
+
+* **`--crd-dir (string)`**
+
+  Directory to load CustomResourceDefinitions from prior to running the tests.
+
+* **`--kind-config (string)`**
+
+  Specify the KIND configuration file path (implies `--start-kind`, cannot be used with `--start-control-plane`).
+
+* **`--kind-context (string)`**
+
+  Specify the KIND context name to use (default: `kind`).
+
+* **`--manifest-dir (stringArray)`**
+
+  One or more directories containing manifests to apply before running the tests.
+
+* **`--parallel (int)`**
+
+  The maximum number of tests to run at once. (default `8`)
+
+* **`--skip-cluster-delete (bool)`**
+
+  If set, do not delete the mocked control plane or kind cluster.
+
+* **`--skip-delete (bool)`**
+
+  If set, do not delete resources created during tests (helpful for debugging test failures, implies `--skip-cluster-delete`).
+
+* **`--start-control-plane (bool)`**
+
+  Start a local Kubernetes control plane for the tests (requires `etcd` and `kube-apiserver` binaries, cannot be used with `--start-kind`).
+
+* **`--start-kind (bool)`**
+
+  Start a KIND cluster for the tests (cannot be used with `--start-control-plane`).
+
+* **`--test (string)`**
+
+  If set, the specific test case to run.
+
+* **`--test-run-labels (string)`**
+
+  Optional label set to associate with this test run.
+  This label set can then be matched against by the `testRunSelector` in `TestFile` objects to optionally exclude selected files.
+  The syntax is comma-separated list of `key=value` assignments.
+
+* **`-v or -vv (int)`**
+  Logging verbosity level. 0=normal, 1=verbose, 2=detailed, 3 or more =trace.
+
+
+
+## Examples
+
+### KUTTL Test
+
+KUTTL test command is the heart of the test harness.  It requires a `kuttl-test.yaml` which defines the test setup.
+
+```yaml
+apiVersion: kuttl.dev/v1beta1
+kind: TestSuite
+testDirs:
+- ./test/integration
+parallel: 4
+```
+
+The default can be run as follows:
+
+```bash
+kubectl kuttl test  pkg/test/test_data/
+```
+
+When running with no defined [test environment](testing/test-environments.md), the default is a preconfigured cluster defined in `$KUBECONFIG`.
+
+To run with the mocked control plane run:
+
+```bash
+kubectl kuttl test --start-control-plane pkg/test/test_data/
+```
+
+In order to run with the full kind cluster stack, run:
+
+```bash
+kubectl kuttl test --start-kind pkg/test/test_data/
+```

docs/contributing.md

@@ -0,0 +1,38 @@
+---
+title: Contributing
+type: docs
+menu: docs
+---
+
+# Contributing
+
+The source code for [KUTTL](https://github.com/kudobuilder/kuttl) lives on GitHub. We welcome feature requests and bug reports in the form of [issues](https://help.github.com/en/articles/about-issues), and of course code - which includes documentation! - in the form of [pull requests](https://help.github.com/en/articles/about-pull-requests) (PRs).
+
+There's a ton of stuff to do and there's opportunities to contribute in a variety of ways. We'd suggest that newcomers look at issues tagged with ['good first issue'](https://github.com/kudobuilder/kuttl/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) and ['help wanted'](https://github.com/kudobuilder/kuttl/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) and then then jump into [#kudo on the Kubernetes Slack](https://kubernetes.slack.com/messages/kudo/) to discuss (join Kubernetes slack via [self-invitation link](https://slack.k8s.io/) if you don't have an account yet).  KUTTL is now separate from KUDO, however we currently plan to use the same communication channels.
+
+Please also take some time to read our [Contributing Guidelines](https://github.com/kudobuilder/kuttl/blob/master/CONTRIBUTING.md).
+
+## Raising an Issue
+
+If you've hit a bug, have an idea for a new feature, or want to suggest some other kind of change then we welcome an issue detailing your problem or your suggestion. Ideally we'd ask that people reach out in the Slack channel and join one of our weekly meetings so that other developers and users can help iterate.
+
+## Creating a Pull Request
+
+Yes please! Bring us your code! There's a whole lot of work to do, and we're committed to building an active community around KUTTL in order to ensure its longevity.
+
+PRs raised against either repo have a default template which help guide contributors to focus on the details necessary for a speedy review. Again, please follow-up with discussion in the Slack channel. We're also happy for people to submit draft PRs which can then be worked through with other members of the KUDO/KUTTL community.
+
+## Reviewing a Pull Request
+
+This process is adapted from the one defined for [contributing to Kubernetes](https://kubernetes.io/docs/contribute/intermediate/#review-a-pr) itself, so should be familiar.
+
+* Examine the PR description and read any associated issues or links for context;
+* Look over all changed files, and if you have a comment or a question on any highlighted section then start a review;
+* Continue to add comments using this review process and when you've finished, choose either 'comment' for general commentary or 'request changes' for anything you deem important enough to warrant further work;
+* If you spot a relatively trivial error such as a typo or something that's not directly related to the stated purpose of the PR then you can let the submitter know by prefixing your review comment with `nit:`. These are not necessarily blockers to the PR itself but it gives the author an opportunity to make amendments;
+* If you think the PR is ready to be merged, then you can add the command `/approve` to your summary comment. Note that only those listed in the approvers section of the [OWNERS](https://github.com/kudobuilder/kudo/blob/master/OWNERS) file can use this command;
+* PRs can be assigned to an individual with the `/assign` command. If you think a proposed change needs a specific person's input, use this command along with their GitHub username to get their attention;
+* If a PR has the `lgtm` and / or the `approve` label then it will be merged automatically;
+  * You can apply the `do-not-merge/hold` label in order to stop PRs from being merged automatically.
+
+Typically, a PR needs a review and an approval from two [core developers](https://github.com/orgs/kudobuilder/people) in order to be merged.