Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instructions for how to test with testing farm #134

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/modules/ROOT/pages/how-tos/_nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
**** xref:how-tos/testing/integration/rerunning.adoc[Rerunning an integration test]
**** xref:how-tos/testing/integration/override-snapshots.adoc[Creating an override snapshot]
**** xref:how-tos/testing/integration/choosing-contexts.adoc[Choosing when to run certain Integration Tests]
**** Third Parties
***** xref:how-tos/testing/integration/third-parties/testing-farm.adoc[Testing with Testing Farm]
Comment on lines +28 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you anticipate having many more of these? I think that this is the first indentation of this length and if feels like it might be too much, especially since there is no content for "Third Parties."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do anticipate more - how to trigger jenkins, how to trigger prow, etc.

** xref:how-tos/metadata/index.adoc[Inspecting provenance and attestations]
*** xref:how-tos/metadata/sboms.adoc[Inspecting SBOMs]
*** xref:how-tos/metadata/attestations.adoc[Inspecting artifact attestations]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
= Creating secrets for your builds
= Creating secrets for your pipelines

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't disagree with the accuracy of this change, but I think it muddies the intention. This is in the section "Configuring your builds" along with all the rest of the build pipeline configurations. Should we move this section out?

I think that these secrets are called build-time secrets in the UI, but that was to differentiate between the secrets that we used to have to deployments.

When you build your pipeline, you might want to add tasks that require **secrets** in order to access external resources.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ NOTE: By default, all integration test scenarios are mandatory and must pass. A

. Select *Add integration test*.

. To start building a new component, either open a new pull request (PR) that targets the tracked branch of the component in the GitHub repository, or comment '/retest' on an existing PR.

.Verification

To start building a new component, either open a new pull request (PR) that targets the tracked branch of the component in the GitHub repository, or comment '/retest' on an existing PR.

When the new build is finished:

. Go to the *Integration tests* tab and select the highlighted name of your test.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
= Testing Farm

In this guide, you'll learn how to xref:/how-tos/testing/integration/adding.adoc[add a custom integration test] in {ProductName} that uses link:https://docs.testing-farm.io/[Testing Farm] as a third-party backend for test execution.

.Prerequisites

. You have xref:/how-tos/creating.adoc[created an application] in {ProductName}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users would need to create an application and a component?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True!


. You have a Testing Farm API key, which can be acquired by following the link:https://docs.testing-farm.io/Testing%20Farm/0.1/onboarding.html[Testing Farm onboarding guide].

. You have have or are ready to create a git repository with link:https://fmf.readthedocs.io/[fmf] files suitable for use with the link:https://tmt.readthedocs.io/[tmt] testing tool. This is the testing technology used by Testing Farm. If you don't have a fmf tests already prepared, you can use our tmt link:https://github.com/ralphbean/tmt-hello-world[hello world repository].

.Procedure

You need to perform two major steps. Upload your Testing Farm API key, and register an *Integration test* to send requests to the Testing Farm API. Complete the following steps in the {ProductName} console:

.Procedure - Upload your Testing Farm API key

Follow the instructions in the xref:/how-tos/configuring/creating-secrets.adoc[creating secrets] guide, with the following details:

. For **Secret name**, enter `testing-farm-secret`.

. Under **Key/value secret**, expand **Key/value 1**, then enter the key name `testing-farm-token`.

NOTE: Be sure that the secret name is `testing-farm-secret` and the key name is `testing-farm-token`. The link:https://github.com/ralphbean/testing-farm-tekton/blob/main/tasks/testing-farm.yaml[tekton task] expects the secret and key name to be named these strings exactly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to move this elsewhere other than in your repo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do, but I want to see if @lukaszachy has better resources I should be directing readers to.


. For **Upload the file with value for your key or paste its contents**, paste the value of your token into the space under **Upload**.

.Procedure - Registering the Integration Test

Follow the instructions in the xref:/how-tos/testing/integration/adding.adoc[adding an integration test] guide, with the following details:

. In the *GitHub URL* field, enter `https://github.com/ralphbean/testing-farm-tekton`, which is a git repository containing a tekton pipeline and task that can send requests to the Testing Farm API.

. In the *Path in repository* field, enter `pipelines/testing-farm.yaml`, which refers to link:https://github.com/ralphbean/testing-farm-tekton/blob/main/pipelines/testing-farm.yaml[this path in the repo].

. Expand the *Parameters* field.

. Select *Add parameter*. For the *Name* field, enter `GIT_URL`. For the *Value* field, enter the url to the git repository that contains your *fmf* files. If you do not have any, try using our tmt link:https://github.com/ralphbean/tmt-hello-world[hello world repository] to try it out.

. Optional: Select *Add parameter*. For the *Name* field, enter `COMPOSE`. For the *Value* field, enter a valid Testing Farm compose identifier. Try `Fedora-40`.

.Verification

To start building a new component, either open a new pull request (PR) that targets the tracked branch of the component in the GitHub repository, or comment '/retest' on an existing PR.

When the new build is finished:

. Go to the *Integration tests* tab and select the highlighted name of your test.

. Go to the *Pipeline runs* tab of that test and select the most recent run.

. You should be able to find the testing-farm URLs in the logs.

. xref:/how-tos/testing/integration/editing.adoc[Edit the integration test] if it is not properly configured.
Loading