Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[kube-prometheus-stack] Default ingress value pathType: ImplementationSpecific does't work on cilium ingress #4228

Open
adippl opened this issue Feb 6, 2024 · 1 comment · May be fixed by #5042
Open

[kube-prometheus-stack] Default ingress value pathType: ImplementationSpecific does't work on cilium ingress #4228

adippl opened this issue Feb 6, 2024 · 1 comment · May be fixed by #5042
Labels
bug Something isn't working

Comments

@adippl
Copy link

adippl commented Feb 6, 2024 •
edited
Loading

Describe the bug a clear and concise description of what the bug is.

prometheus-kube-stack by default deploys prometheus ingress with pathType: ImplementationSpecific.
This this doesn't work with cilium ingress because ImplementationSpecific pathType acts as exact match.
Changing setting value pathType: Prefix solves the issue creates working ingress.
Please consider changing the default values or add a warning talking about this issue in the comments of values file.

The same issue in cilium repo. cilium/cilium#30119

What's your helm version?

version.BuildInfo{Version:"v3.11", GitCommit:"", GitTreeState:"", GoVersion:"go1.21.5"}

What's your kubectl version?

Client Version: v1.28.5 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.2

Which chart?

prometheus-kube-stack

What's the chart version?

56.6.2

What happened?

No response

What you expected to happen?

No response

How to reproduce it?

No response

Enter the changed values of values.yaml?

fullnameOverride: "kps"
prometheus:
  service:
    type: LoadBalancer
  networkPolicy:
    enabled: false

    flavor: kubernetes
  ingress:
    enabled: true
    ingressClassName: cilium
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
    labels: {}
    hosts:
      - prometheus.DOMAIN
    path: /.*
    tls:
    - hosts:
        - prometheus.DOMAIN
      secretName: prometheus.DOMAIN
    pathType: Prefix
  prometheusSpec:
    priorityClassName: "high-priority"
    externalLabels:
      cluster: k8s3
    retention: 30d
    replicas: 1
    podAntiAffinity: "hard"
    storageSpec:
      volumeClaimTemplate:
        spec:
          storageClassName: rook-ceph-block-ssd
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 25Gi
    resources:
     requests:
       cpu: 100m
       memory: 768Mi
     limits:
       cpu: 1000m
       memory: 2048Mi
    priorityClassName: "high-priority"

Enter the command that you execute and failing/misfunctioning.

curl https://prometheus.<<<MYDOMAIN>>>/graph -v
*   Trying 10.0.5.98:443...
* Connected to prometheus.<<<MYDOMAIN>>> (10.0.5.98) port 443
* ALPN: curl offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=prometheus.<<<MYDOMAIN>>>
*  start date: Jan  4 17:58:15 2024 GMT
*  expire date: Apr  3 17:58:14 2024 GMT
*  subjectAltName: host "prometheus.<<<MYDOMAIN>>>" matched cert's "prometheus.<<<MYDOMAIN>>>"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* using HTTP/1.x
> GET /graph HTTP/1.1
> Host: prometheus.<<<MYDOMAIN>>>
> User-Agent: curl/8.4.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 404 Not Found
< date: Fri, 05 Jan 2024 14:24:01 GMT
< server: envoy
< content-length: 0
< 
* Connection #0 to host prometheus.<<<MYDOMAIN>>> left intact

Anything else we need to know?

No response
@adippl adippl added the bug Something isn't working label Feb 6, 2024
@zeritti zeritti changed the title [prometheus-kube-stack] Default ingress value pathType: ImplementationSpecific does't work on cilium ingress [kube-prometheus-stack] Default ingress value pathType: ImplementationSpecific does't work on cilium ingress Feb 8, 2024
@pierreozoux
Copy link

pierreozoux commented Dec 6, 2024 •
edited
Loading

The ingress object looks like this:

    - host: alertmanager.mycluster.net
        paths:
          path: /
          pathType: ImplementationSpecific

I really think it could be replaced with Prefix without troubles.
It would also help to enforce mitigation for this CVE-2022-4886, it is recommended to forbid ImplementationSpecific as a cluster admin.

relates to cert-manager/cert-manager#6805 (comment)

For instance, this is the fix commit in OpenStack.

pierreozoux added a commit to pierreozoux/prometheus-community-helm-charts that referenced this issue Dec 6, 2024
@pierreozoux
fix: use prefix pathType
1c80242 
fixes prometheus-community#4228

and address recomendation written in CVE-2022-4886 -  kubernetes/kubernetes#126815

Signed-off-by: Pierre Ozoux <[email protected]>
This was referenced Dec 6, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: use prefix pathType pierreozoux/prometheus-community-helm-charts
2 participants
@pierreozoux @adippl