Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve readability of test output #20

Open
johncblandii opened this issue Apr 23, 2020 · 0 comments
Open

Improve readability of test output #20

johncblandii opened this issue Apr 23, 2020 · 0 comments

Comments

@johncblandii
Copy link
Contributor

➜ variant test
terraform init
terraform plan -out=acme.planfile
aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm
case "ok": assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
, where run.res.stdout=aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm (string) case.out=aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm (string)aws --profile company-blah-helm eks update-kubeconfig --name=company-blah-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah-helm
case "ok2": assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
, where run.res.stdout=aws --profile company-blah-helm eks update-kubeconfig --name=company-blah-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah-helm (string) case.out=aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm (string)--- FAIL: aws_eks_kubeconfig (0.01s)
    --- FAIL: aws_eks_kubeconfig/ok (0.01s)
        app.go:1040: case "ok": assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
            , where run.res.stdout=aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm (string) case.out=aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm (string)
    --- FAIL: aws_eks_kubeconfig/ok2 (0.00s)
        app.go:1040: case "ok2": assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
            , where run.res.stdout=aws --profile company-blah-helm eks update-kubeconfig --name=company-blah-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah-helm (string) case.out=aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm (string)
FAIL
Error: test exited with code 1

Note: the failure here is fine. I was refactoring.

This output is not very readable.

  • app.go:1040: is irrelevant to my tests
  • no spaces between the case failures makes it one huge blog
  • case failures output as a long line
  • stdout shows from the internal CLI calls (I'm using an echo so it shows what would run so ignore that part), but that may not always be desired
  • Adding some color would be phenomenal to readability

Potential output without errors:

➜ variant test
PASS terraform-core_test.variant
  ✓  case "init" (0.01s): terraform init
  ✓  case "plan" (0.01s): terraform plan -out=acme.planfile

PASS aws_test.variant
  ✓  case "ok" (0.01s): aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm
  ✓  case "ok2" (0.01s): aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm

Potential output with errors:

➜ variant test
PASS terraform-core_test.variant
  ✓  case "init" (0.01s): terraform init
  ✓  case "plan" (0.01s): terraform plan -out=acme.planfile

FAIL aws_test.variant

  ✕  case "ok" (0.01s): aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm

  assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
  run.res.stdout
    (string) aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm
  case.out
    (string) aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm

  ✕  case "ok2" (0.01s): aws --profile company-blah-helm eks update-kubeconfig --name=company-blah-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah-helm

  assertion "out" failed: this expression must be true, but was false: run.res.stdout == case.out
  run.res.stdout
    (string) aws --profile company-blah2-helm eks update-kubeconfig --name=company-blah2-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah2-helm (string) case.out=aws --profile company-acme-helm eks update-kubeconfig --name=company-acme-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.acme-helm
  case.out
    (string) aws --profile company-blah-helm eks update-kubeconfig --name=company-blah-eks-cluster --region=us-east-2 --kubeconfig=/path/to/kube/kubecfg.blah-helm

Error: test exited with code 1

Note: this one is still pretty verbose and could likely be cleaned up a bit, but it is readable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant