Not able to generate configauditreport from trivy-operator on openshift cluster

[umeshvw]

I have installed trivy-operator(both v0.23.0 and v21.4) on openshift cluster( cluster version: 4.17.9).
All pods are in running state and I am getting vulnerability report and other reports except configauditreport. I tried to check for logs but could not find any particular issues.

I suspect there is some issue with CRD's installed ( not able to attach the yaml

PFB

C:\Users>oc get pods -o wide -n trivy-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
node-collector-6c4b84cbbf-5lc24 0/1 Completed 0 3d 10.131.6.42
scan-vulnerabilityreport-5995b59d86-brmmb 3/3 Running 0 18s 10.131.12.211
scan-vulnerabilityreport-689458bf4d-62k8w 0/1 Running 0 7s 10.130.49.204
scan-vulnerabilityreport-69b8567c4b-qjx8m 0/3 Running 0 8s 10.131.79.70
trivy-operator-9fb88dbfc-qkg6q 1/1 Running 0 2d23h 10.131.12.77

$ oc get configauditreports -A
No resources found

**What did you expect to happen: Trivy operator should publish configauditreport

Environment:

• Trivy-Operator version (use trivy-operator version): V0.23.0 and v0.21.4( i tried both)
• Kubernetes version (use kubectl version): openshift 4.17.9
• OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): ubuntu 22

I am adding CRD yaml here as was able to attach the yaml directly:

clusterconfigauditreports.aquasecurity.github.io:

kind: CustomResourceDefinition
apiVersion: apiextensions.k8s.io/v1
metadata:
name: clusterconfigauditreports.aquasecurity.github.io
uid: 67eb3e7f-3b65-4c50-82b2-9a3313a68518
resourceVersion: '2427179094'
generation: 1
creationTimestamp: '2025-01-17T08:15:19Z'
annotations:
controller-gen.kubebuilder.io/version: v0.14.0
- manager: kube-apiserver
operation: Update
apiVersion: apiextensions.k8s.io/v1
time: '2025-01-17T08:15:19Z'
spec:
group: aquasecurity.github.io
names:
plural: clusterconfigauditreports
singular: clusterconfigauditreport
shortNames:
- clusterconfigaudit
kind: ClusterConfigAuditReport
listKind: ClusterConfigAuditReportList
scope: Cluster
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
description: ClusterConfigAuditReport is a specification for the ClusterConfigAuditReport resource.
type: object
required:
- report
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
report:
type: object
required:
- checks
properties:
checks:
description: Checks provides results of conducting audit steps.
type: array
items:
description: Check provides the result of conducting a single audit step.
type: object
required:
- checkID
- severity
- success
properties:
success:
type: boolean
checkID:
type: string
remediation:
description: Remediation provides description or links to external resources to remediate failing check.
type: string
messages:
type: array
items:
type: string
scope:
description: Scope indicates the section of config that was audited.
type: object
required:
- type
- value
properties:
type:
description: 'Type indicates type of this scope, e.g. Container, ConfigMapKey or JSONPath.'
type: string
value:
description: 'Value indicates value of this scope that depends on Type, e.g. container name, ConfigMap key or JSONPath expression'
type: string
title:
type: string
description:
type: string
severity:
description: Severity level of a vulnerability or a configuration audit check.
type: string
category:
type: string
scanner:
description: Scanner is the spec for a scanner generating a security assessment report.
type: object
required:
- name
- vendor
- version
properties:
name:
description: Name the name of the scanner.
type: string
vendor:
description: Vendor the name of the vendor providing the scanner.
type: string
version:
description: Version the version of the scanner.
type: string
summary:
description: ConfigAuditSummary counts failed checks by severity.
type: object
required:
- criticalCount
- highCount
- lowCount
- mediumCount
properties:
criticalCount:
description: CriticalCount is the number of failed checks with critical severity.
type: integer
highCount:
description: HighCount is the number of failed checks with high severity.
type: integer
lowCount:
description: LowCount is the number of failed check with low severity.
type: integer
mediumCount:
description: MediumCount is the number of failed checks with medium severity.
type: integer
updateTimestamp:
type: string
format: date-time
x-kubernetes-preserve-unknown-fields: true
subresources: {}
additionalPrinterColumns:
- name: Scanner
type: string
description: The name of the config audit scanner
jsonPath: .report.scanner.name
- name: Age
type: date
description: The age of the report
jsonPath: .metadata.creationTimestamp
- name: Critical
type: integer
description: The number of failed checks with critical severity
priority: 1
jsonPath: .report.summary.criticalCount
- name: High
type: integer
description: The number of failed checks with high severity
priority: 1
jsonPath: .report.summary.highCount
- name: Medium
type: integer
description: The number of failed checks with medium severity
priority: 1
jsonPath: .report.summary.mediumCount
- name: Low
type: integer
description: The number of failed checks with low severity
priority: 1
jsonPath: .report.summary.lowCount
conversion:
strategy: None
status:
conditions:
- type: NamesAccepted
status: 'True'
lastTransitionTime: '2025-01-17T08:15:19Z'
reason: NoConflicts
message: no conflicts found
- type: Established
status: 'True'
lastTransitionTime: '2025-01-17T08:15:19Z'
reason: InitialNamesAccepted
message: the initial names have been accepted
acceptedNames:
plural: clusterconfigauditreports
singular: clusterconfigauditreport
shortNames:
- clusterconfigaudit
kind: ClusterConfigAuditReport
listKind: ClusterConfigAuditReportList
storedVersions:
- v1alpha1

clustercompliancereports.aquasecurity.github.io:

kind: CustomResourceDefinition
apiVersion: apiextensions.k8s.io/v1
metadata:
name: clustercompliancereports.aquasecurity.github.io
uid: 73b40d5e-6288-42da-a404-e284ac906270
resourceVersion: '2427179079'
generation: 1
creationTimestamp: '2025-01-17T08:15:19Z'
annotations:
controller-gen.kubebuilder.io/version: v0.14.0
spec:
group: aquasecurity.github.io
names:
plural: clustercompliancereports
singular: clustercompliancereport
shortNames:
- compliance
kind: ClusterComplianceReport
listKind: ClusterComplianceReportList
scope: Cluster
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
description: ClusterComplianceReport is a specification for the ClusterComplianceReport resource.
type: object
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: ReportSpec represent the compliance specification
type: object
required:
- compliance
- cron
- reportType
properties:
compliance:
type: object
required:
- controls
- description
- id
- relatedResources
- title
- version
properties:
controls:
description: Control represent the cps controls data and mapping checks
type: array
items:
description: Control represent the cps controls data and mapping checks
type: object
required:
- id
- name
- severity
properties:
checks:
type: array
items:
description: SpecCheck represent the scanner who perform the control check
type: object
required:
- id
properties:
id:
description: id define the check id as produced by scanner
type: string
defaultStatus:
description: define the default value for check status in case resource not found
type: string
enum:
- PASS
- WARN
- FAIL
description:
type: string
id:
description: id define the control check id
type: string
name:
type: string
severity:
description: define the severity of the control
type: string
enum:
- CRITICAL
- HIGH
- MEDIUM
- LOW
- UNKNOWN
description:
type: string
id:
type: string
relatedResources:
type: array
items:
type: string
title:
type: string
version:
type: string
cron:
description: cron define the intervals for report generation
type: string
pattern: '^(((([*]{1}){1})|((*/){0,1}(([0-9]{1}){1}|(([1-5]{1}){1}([0-9]{1}){1}){1}))) ((([*]{1}){1})|((*/){0,1}(([0-9]{1}){1}|(([1]{1}){1}([0-9]{1}){1}){1}|([2]{1}){1}([0-3]{1}){1}))) ((([*]{1}){1})|((*/){0,1}(([1-9]{1}){1}|(([1-2]{1}){1}([0-9]{1}){1}){1}|([3]{1}){1}([0-1]{1}){1}))) ((([*]{1}){1})|((*/){0,1}(([1-9]{1}){1}|(([1-2]{1}){1}([0-9]{1}){1}){1}|([3]{1}){1}([0-1]{1}){1}))|(jan|feb|mar|apr|may|jun|jul|aug|sep|okt|nov|dec)) ((([*]{1}){1})|((*/){0,1}(([0-7]{1}){1}))|(sun|mon|tue|wed|thu|fri|sat)))$'
reportType:
type: string
enum:
- summary
- all
status:
type: object
required:
- updateTimestamp
properties:
detailReport:
description: ComplianceReport represents a kubernetes scan report
type: object
properties:
description:
type: string
id:
type: string
relatedVersion:
type: array
items:
type: string
results:
type: array
items:
type: object
required:
- checks
properties:
checks:
type: array
items:
description: ComplianceCheck provides the result of conducting a single compliance step.
type: object
required:
- checkID
- severity
- success
properties:
success:
type: boolean
checkID:
type: string
remediation:
description: Remediation provides description or links to external resources to remediate failing check.
type: string
messages:
type: array
items:
type: string
target:
type: string
title:
type: string
description:
type: string
severity:
description: Severity level of a vulnerability or a configuration audit check.
type: string
category:
type: string
description:
type: string
id:
type: string
name:
type: string
severity:
type: string
status:
type: string
title:
type: string
version:
type: string
x-kubernetes-preserve-unknown-fields: true
summary:
type: object
properties:
failCount:
type: integer
passCount:
type: integer
summaryReport:
description: SummaryReport represents a kubernetes scan report with consolidated findings
type: object
properties:
controlCheck:
type: array
items:
type: object
properties:
id:
type: string
name:
type: string
severity:
type: string
totalFail:
type: integer
id:
type: string
title:
type: string
x-kubernetes-preserve-unknown-fields: true
updateTimestamp:
type: string
format: date-time
x-kubernetes-preserve-unknown-fields: true
subresources:
status: {}
additionalPrinterColumns:
- name: Age
type: date
description: The age of the report
jsonPath: .metadata.creationTimestamp
- name: Fail
type: integer
description: The number of checks that failed
priority: 1
jsonPath: .status.summary.failCount
- name: Pass
type: integer
description: The number of checks that passed
priority: 1
jsonPath: .status.summary.passCount
conversion:
strategy: None
status:
conditions:
- type: NamesAccepted
status: 'True'
lastTransitionTime: '2025-01-17T08:15:19Z'
reason: NoConflicts
message: no conflicts found
- type: Established
status: 'True'
lastTransitionTime: '2025-01-17T08:15:19Z'
reason: InitialNamesAccepted
message: the initial names have been accepted
acceptedNames:
plural: clustercompliancereports
singular: clustercompliancereport
shortNames:
- compliance
kind: ClusterComplianceReport
listKind: ClusterComplianceReportList
storedVersions:
- v1alpha1

[afdesk]

@umeshvw thanks for your interest

could you check these ways? https://aquasecurity.github.io/trivy-operator/latest/getting-started/installation/troubleshooting/#operator-does-not-create-vulnerabilityreports

and then please share the logs.

[umeshvw]

@afdesk

PFB

$ oc get clusterrolebinding |grep -i trivy-operator
trivy-operator ClusterRole/trivy-operator 3d22h
VW+D0DLRMB@VWVITHLPUB6423 MINGW64 ~
$ oc describe clusterrole trivy-operator
Name: trivy-operator
Labels: app.kubernetes.io/managed-by=Helm
Annotations: meta.helm.sh/release-name: trivy-operator
meta.helm.sh/release-namespace: trivy-system

PolicyRule:
Resources Non-Resource URLs Resource Names Verbs

---

clustercompliancedetailreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clustercompliancereports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clusterconfigauditreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clusterinfraassessmentreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clusterrbacassessmentreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clustersbomreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
clustervulnerabilityreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
configauditreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
exposedsecretreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
infraassessmentreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
rbacassessmentreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
sbomreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
vulnerabilityreports.aquasecurity.github.io [] [] [create delete get list patch update watch]
jobs.batch [] [] [create delete get list watch]
secrets [] [] [create get update]
configmaps [] [] [get list watch]
limitranges [] [] [get list watch]
nodes [] [] [get list watch]
pods [] [] [get list watch]
replicationcontrollers [] [] [get list watch]
resourcequotas [] [] [get list watch]
services [] [] [get list watch]
customresourcedefinitions.apiextensions.k8s.io [] [] [get list watch]
deploymentconfigs.apps.openshift.io [] [] [get list watch]
daemonsets.apps [] [] [get list watch]
deployments.apps [] [] [get list watch]
replicasets.apps [] [] [get list watch]
statefulsets.apps [] [] [get list watch]
cronjobs.batch [] [] [get list watch]
ingresses.networking.k8s.io [] [] [get list watch]
networkpolicies.networking.k8s.io [] [] [get list watch]
clusterrolebindings.rbac.authorization.k8s.io [] [] [get list watch]
clusterroles.rbac.authorization.k8s.io [] [] [get list watch]
rolebindings.rbac.authorization.k8s.io [] [] [get list watch]
roles.rbac.authorization.k8s.io [] [] [get list watch]
pods/log [] [] [get list]
clustercompliancereports.aquasecurity.github.io/status [] [] [get patch update]
nodes/proxy [] [] [get]
serviceaccounts [] [] [get]

These are clusterrole permission to the service account. I was able to create a sample configauditreport in my namespace but it is not getting generated by trivy-operator. trivy-operator is able to generate other reports such as vulnerability reports etc but getting an issue for config audit report generation by trivy-operator

[umeshvw]

@afdesk I am also seeing below error in trivy-operator pod. can you please help.

{"level":"error","ts":"2025-01-20T21:42:43Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2025-01-20T21:49:16Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2025-01-20T21:55:49Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Eval\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:199\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.evaluate\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/helper.go:45\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func11\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:229\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2025-01-20T22:02:22Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2025-01-20T22:08:55Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Eval\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:199\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.evaluate\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/helper.go:45\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func2\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:229\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}

[afdesk]

runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controlle

it looks like you use old version,
my suggestion here is that it depends on old Trivy. and trivy policies (checks) isn't compatible with this one.

Could you rechech with the latest version?

[umeshvw]

@afdesk I tried with latest version but still the same issue.
I had deployed v0.21.4 on another cluster(Dev) where I am getting both vulnerability reports and configauditreports as well.

Earlier i used to get both the reports on Issue cluster with trivy operator version v0.21.4 but then i tried to reinstall by uninstalling helm release and then deleting all vulnerability reports and config audit reports , CRDS manually from the cluster. After this, I am not getting configauditreports on this cluster. Now i have removed everything and then did the installation of v0.23.0 where still I am not getting configauditreports. do you know what might be the issue.

Still i am getting below issue:

{"level":"error","ts":"2025-01-29T14:10:54Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*PolicyConfigController).SetupWithManager.(*PolicyConfigController).reconcileConfig.func5\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/policyconfig.go:129\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2025-01-29T14:17:28Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t Get "https://ghcr.io/v2/\": dial tcp 4.208.26.196:443: connect: connection timed out\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:61\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*PolicyConfigController).SetupWithManager.(*PolicyConfigController).reconcileConfig.func5\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/policyconfig.go:129\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.2/pkg/internal/controller/controller.go:222"}

I have checked in other cluster and it is using ontroller-runtime@v0.17.2

[afdesk]

@umeshvw thanks for the report
I managed to get a similar error locally, I think the main reason is that trivy-operator can't download trivy-check database, right?
in your log there is next message:

"error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get "[https://ghcr.io/v2/\](https://ghcr.io/v2/%5C)": dial tcp 4.208.26.196:443: connect: connection timed out\n\n",

Could you check the connection or use another registry? for example mirror.gcr.io

[umeshvw]

@afdesk Thanks for reply.
Can you please guide me how to use another registry. I see value of policies.bundle.oci.ref in config map "trivy-operator" is set as " ghcr.io/aquasecurity/trivy-checks:0" .do i need to change something here.

This is deployed on OpenShift where we have whitelisted all required URL(github.com, ghcr.io) from OpenShift

[afdesk]

oh, i understood... it looks like you use old chart.

could you try to install Trivy operator via clone the chart directory (https://aquasecurity.github.io/trivy-operator/latest/getting-started/installation/helm/)

$ git clone --depth 1 https://github.com/aquasecurity/trivy-operator.git
$ cd trivy-operator
$ helm install trivy-operator ./deploy/helm \
     --namespace trivy-system \
     --create-namespace 

[umeshvw]

@afdesk Thanks. I am able to get the configauditreports and vulnerabilityreports after deploying above but scan-vulnerability pods are in error state. I am able to see below error message in operator pod logs.

{"level":"error","ts":"2025-01-30T19:02:01Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-867cd8cb5f","container":"4ac6ea61-fb4f-4584-8bb6-fe7bf5fd7022","status.reason":"Error","status.message":"2025-01-30T19:01:58Z\tINFO\tAdding schema version to the DB repository for backward compatibility\trepository="mirror.gcr.io/aquasec/trivy-db:2"\n2025-01-30T19:01:58Z\tINFO\t[vulndb] Need to update DB\n2025-01-30T19:01:58Z\tINFO\t[vulndb] Downloading vulnerability DB...\n2025-01-30T19:01:58Z\tINFO\t[vulndb] Downloading artifact...\trepo="mirror.gcr.io/aquasec/trivy-db:2"\n2025-01-30T19:01:58Z\tFATAL\tFatal error\tinit error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from mirror.gcr.io/aquasec/trivy-db:2: oci download error: failed to fetch the layer: GET https://mirror.gcr.io/v2/aquasec/trivy-db/blobs/sha256:9d0741b215ae2928e19c9af8dc47a6fa2df687ba5f9c2e4dc28101a06424169a: BLOB_UNKNOWN: Unknown blob: sha256:9d0741b215ae2928e19c9af8dc47a6fa2df687ba5f9c2e4dc28101a06424169a\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.TypedFunc[...].Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/reconcile/reconcile.go:124\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"}
{"level":"error","ts":"2025-01-30T19:02:20Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-867cd8cb5f","container":"ba8f5f20-76bc-4fad-8bc6-17c41f2d22e5","status.reason":"Error","status.message":"2025-01-30T19:02:17Z\tINFO\tAdding schema version to the DB repository for backward compatibility\trepository="mirror.gcr.io/aquasec/trivy-db:2"\n2025-01-30T19:02:17Z\tINFO\t[vulndb] Need to update DB\n2025-01-30T19:02:17Z\tINFO\t[vulndb] Downloading vulnerability DB...\n2025-01-30T19:02:17Z\tINFO\t[vulndb] Downloading artifact...\trepo="mirror.gcr.io/aquasec/trivy-db:2"\n2025-01-30T19:02:17Z\tFATAL\tFatal error\tinit error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from mirror.gcr.io/aquasec/trivy-db:2: oci download error: failed to fetch the layer: GET https://mirror.gcr.io/v2/aquasec/trivy-db/blobs/sha256:9d0741b215ae2928e19c9af8dc47a6fa2df687ba5f9c2e4dc28101a06424169a: BLOB_UNKNOWN: Unknown blob: sha256:9d0741b215ae2928e19c9af8dc47a6fa2df687ba5f9c2e4dc28101a06424169a\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.TypedFunc[...].Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/reconcile/reconcile.go:124\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"}

[afdesk]

@umeshvw thanks for the report.
It looks like a known issue in Trivy aquasecurity/trivy#8223

the fix will be included in the next release or you can update Trivy version in your helm chart yourself.

[umeshvw]

@afdesk Thanks. May I know which version I need to update in helm charts

[afdesk]

@afdesk Thanks. May I know which version I need to update in helm charts

I'd try the latest version - v0.59.0.
there is a PR with such update, but It'll be added in the next Trivy-operator release: #2406
but you can see there needed updates