Skip to content

🌱 Add test-operator with custom controller #2070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

🌱 Add test-operator with custom controller #2070

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
17df817
Update test-operator v1.0.0
Jul 8, 2025
f837ca4
Update test-operator v2.0.0
Jul 8, 2025
13a56d3
Move v2.0.0 e2e test to v1.3.0 due to lack of webhook support
Jul 8, 2025
2396e27
Add controller image build and publish support to test registry
Jul 8, 2025
8564214
Add test-operator unit tests to CI
Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/testdata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: testdata

on:
workflow_dispatch:
pull_request:
merge_group:
push:
branches:
- main

jobs:
unit-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-go@v5
with:
go-version-file: go.mod

- name: Run test-operator unit tests
run: |
make test-test-operator
15 changes: 14 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,11 +246,24 @@ test-unit: $(SETUP_ENVTEST) envtest-k8s-bins #HELP Run the unit tests
$(UNIT_TEST_DIRS) \
-test.gocoverdir=$(COVERAGE_UNIT_DIR)


.PHONY: test-test-operator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a function of the Operator/Mock sample itself.
It should not be part of the operator-controller Makefile at all — it belongs inside the sample.

Placing it in the main Makefile goes against important separation-of-concerns principles.
It introduces coupling that will make these components harder to maintain over time.
Keeping this logic self-contained in the sample is key to preserving clarity and long-term maintainability.

test-test-operator: $(SETUP_ENVTEST) envtest-k8s-bins
KUBEBUILDER_ASSETS="$(shell $(SETUP_ENVTEST) use -p path $(ENVTEST_VERSION) $(SETUP_ENVTEST_BIN_DIR_OVERRIDE))" \
CGO_ENABLED=1 go test -tags '$(GO_BUILD_TAGS)' -count=1 -race -short ./testdata/images/bundles/test-operator/...
Copy link
Contributor

@camilamacedo86 camilamacedo86 Jul 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! 👋

I think we should mock the scenarios just like our users would experience them. This means:

  • The flow to generate a project and a bundle should mirror what a real user does. ( create project with SDK or Kubebuilder, inject the code, run the commands and generate a BUNDLE )
  • We should have a script that automates those steps, so it's repeatable.

Therefore, we can keep our samples well-maintained.

Now, doing all this at once might be a bit much, so I’m totally fine starting with fixed samples, and then we can evolve toward full automation in follow-ups and discuss all properly.

However: We need to change the file structure:

We can’t organize it as bundles/test-operator | versions because:

  1. We don’t have bundles — we have the actual projects.
  2. The current path is strange — why are we putting projects under images/?
  3. ⚠️ The directory name matters — tools like Kubebuilder or the SDK treat that as the project name. So if we use v1.0.0 as the folder name, automation breaks.

Proposal

Could we change it here to be like:

testdata/operators/
  ├── v1.0.0/
  │    └── test-operator/
  ├── v1.3.0/
  │    └── test-operator/
  └── v2.0.0/
       └── test-operator/

This way:

  • Each version gets its own isolated test setup.
  • The project directory stays named test-operator, so tooling works as expected.
  • It’s clean, easy to automate, and mirrors user behavior more closely.

Let me know what you think!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By following my suggestion here for we have a real Operator that can be maintained via automation #2070 (comment), could we please use a name like memcached-operator instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda get what you're trying to do, but I don't think that's scalable or maintainable given the necessary controller, webhook, and crd code that will be required by the test-operators (also between versions of it), which will either require patching or copying over generated code, which can be brittle, error prone, and create problems that are annoying to debug and fix - the only benefit being that we'll know when the tools no longer work for building the fixtures.

The point of the test-operator isn't to exercise the steps involved in creating an operator and to make sure that works with OLMv1. We already have a test that exercises this developer flow. The point is to have a small, simple, test-operator which we own and can be used to exercise different features through out our e2e tests.

I'd prefer not to call it memcached-operator since it isn't. What's wrong with test-operator?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The raised problem here is with the path that does not allow we have those samples well maintained, well documented and well generated in the long term, and the names does not reflect to the reality, could we

testdata/operators/
  ├── v1.0.0/
  │    └── test-operator/
  ├── v1.3.0/
  │    └── test-operator/
  └── v2.0.0/
       └── test-operator/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why they can't be well maintained. It seems more natural to group the versions under the operator than the operator under the versions...?


TEST_OPERATOR_CONTROLLERS_HOME=./testdata/images/controllers
TEST_OPERATOR_CONTROLLERS=v1.0.0 v2.0.0
Copy link
Contributor

@tmshort tmshort Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this missing v1.3.0? Or is it not needed because it's not actually built?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, this is just for building the controller binaries. The 1.3.0 version uses the 1.0.0 controller.

Copy link
Contributor

@camilamacedo86 camilamacedo86 Jul 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I did not understood it as well because

  • We need to have the "operator"
  • Then, we need run the commands from this project to generate the manager (it will be image)
  • Then, we need generate the bundle for each case scenario (using SDK or OPM for example)
  • Then, we need add those bundles to a catalogue
  • Then, create a Catalog Source for those

The 1.3.0 version uses the 1.0.0 controller
we have a verion that we use the bin generated by another?
I did not understood it

Copy link
Contributor Author

@perdasilva perdasilva Jul 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is related to the currently existing e2e tests. The v1.3.0 bundle in the PR substitutes the v2.0.0 bundle in main [this] commit.

There's only a change in the included manifests (namely the configmap) between v1.0.0 and v1.3.0. I.e. v1.3.0 still uses the same controller as v1.0.0. This make target builds only the controllers. The bundle image itself is built in testdata/push - which is how it's currently done anyway.


.PHONY: $(TEST_OPERATOR_CONTROLLERS)
$(TEST_OPERATOR_CONTROLLERS):
go build $(GO_BUILD_FLAGS) $(GO_BUILD_EXTRA_FLAGS) -tags '$(GO_BUILD_TAGS)' -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o $(TEST_OPERATOR_CONTROLLERS_HOME)/test-operator/$@/manager ./testdata/images/bundles/test-operator/$@/cmd/main.go

.PHONY: image-registry
E2E_REGISTRY_IMAGE=localhost/e2e-test-registry:devel
image-registry: export GOOS=linux
image-registry: export GOARCH=amd64
image-registry: ## Build the testdata catalog used for e2e tests and push it to the image registry
image-registry: $(TEST_OPERATOR_CONTROLLERS) ## Build the testdata catalog used for e2e tests and push it to the image registry
go build $(GO_BUILD_FLAGS) $(GO_BUILD_EXTRA_FLAGS) -tags '$(GO_BUILD_TAGS)' -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o ./testdata/push/bin/push ./testdata/push/push.go
$(CONTAINER_RUNTIME) build -f ./testdata/Dockerfile -t $(E2E_REGISTRY_IMAGE) ./testdata
$(CONTAINER_RUNTIME) save $(E2E_REGISTRY_IMAGE) | $(KIND) load image-archive /dev/stdin --name $(KIND_CLUSTER_NAME)
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/cluster_extension_install_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -732,7 +732,7 @@ func TestClusterExtensionInstallReResolvesWhenCatalogIsPatched(t *testing.T) {
assert.Equal(ct, metav1.ConditionTrue, cond.Status)
assert.Equal(ct, ocv1.ReasonSucceeded, cond.Reason)
assert.Contains(ct, cond.Message, "Installed bundle")
assert.Contains(ct, clusterExtension.Status.Install.Bundle.Version, "2.0.0")
assert.Contains(ct, clusterExtension.Status.Install.Bundle.Version, "1.3.0")
}
}, pollDuration, pollInterval)

Expand Down
1 change: 1 addition & 0 deletions testdata/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
push/bin
images/controllers
36 changes: 36 additions & 0 deletions testdata/images/bundles/test-operator/v1.0.0/api/v1/groupversion_info.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
Copyright 2025.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// Package v1 contains API Schema definitions for the testolm v1 API group.
// +kubebuilder:object:generate=false
// +groupName=testolm.operatorframework.io
package v1

import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
)

var (
// GroupVersion is group version used to register these objects.
GroupVersion = schema.GroupVersion{Group: "testolm.operatorframework.io", Version: "v1"}

// SchemeBuilder is used to add go types to the GroupVersionKind scheme.
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}

// AddToScheme adds the types in this group-version to the given scheme.
AddToScheme = SchemeBuilder.AddToScheme
)
57 changes: 57 additions & 0 deletions testdata/images/bundles/test-operator/v1.0.0/api/v1/testoperator_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
Copyright 2025.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// TestOperatorSpec defines the desired state of TestOperator.
type TestOperatorSpec struct {
// +optional
Message string `json:"message,omitempty"`
}

// TestOperatorStatus defines the observed state of TestOperator.
type TestOperatorStatus struct {
Echo string `json:"echo,omitempty"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status

// TestOperator is the Schema for the testoperators API.
type TestOperator struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec TestOperatorSpec `json:"spec,omitempty"`
Status TestOperatorStatus `json:"status,omitempty"`
}

// +kubebuilder:object:root=true

// TestOperatorList contains a list of TestOperator.
type TestOperatorList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []TestOperator `json:"items"`
}

func init() {
SchemeBuilder.Register(&TestOperator{}, &TestOperatorList{})
}
114 changes: 114 additions & 0 deletions testdata/images/bundles/test-operator/v1.0.0/api/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading