Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LogOutputPaths not working #7718

Open
vijayck2020 opened this issue Feb 10, 2025 · 7 comments
Open

LogOutputPaths not working #7718

vijayck2020 opened this issue Feb 10, 2025 · 7 comments
Assignees
Labels
bug Something isn't working good-first-issue Good for newcomers help-wanted Extra attention is needed triage/needs-investigation Issues that need to be investigated before triaging

Comments

@vijayck2020
Copy link

vijayck2020 commented Feb 10, 2025

  • name: LOG_LEVEL
    value: info
  • name: LOG_OUTPUT_PATHS
    value: "/var/log/controller.log"
  • name: LOG_ERROR_OUTPUT_PATHS
    value: stderr
  • name: METRICS_PORT
    value: "8000"

Since I am running karpenter in a fargate node, configured a ephemeral volume to be shared between controller and fluentbit containers. But the karpenter is not writing any logs to the configured file. Pasted above is the respective section in my deployment

In my values.yml file, I have given the below values

port: 8080
healthProbe:
port: 8081
webhook:
enabled: true
port: 8443
metrics:
port: 8001
logLevel: info
logOutputPaths:
stdout
/var/controller.log
logErrorOutputPaths:
stderr

I am using version 1.0.6 and karpenter is not writing any logs at all to the file I have provided. Still karpenter is logging to the stdout only.
Can someone take a look and give some valuable feedbacks?
Thanks in advance

@vijayck2020
Copy link
Author

karpenter-values.txt

This is the current values file I am using

@jonathan-innis jonathan-innis added help-wanted Extra attention is needed bug Something isn't working triage/needs-investigation Issues that need to be investigated before triaging labels Feb 13, 2025
@jonathan-innis
Copy link
Contributor

This is really an upstream change with the logging config if that's broken in some way -- I think we just need someone to dig into the code and quickly validate that the parsing is working as expected here and configured correctly in the Zap config

@jonathan-innis jonathan-innis added the good-first-issue Good for newcomers label Feb 13, 2025
@liafizan
Copy link
Contributor

Could you please assign this to me?

@liafizan
Copy link
Contributor

@vijayck2020 on my first pass, I am unable to reproduce this issue. I am not at the same version as yours but this feature was added in v1 and using that version.

I mounted a empty dir and configured karpenter as below:

        - name: LOG_OUTPUT_PATHS
          value: stdout,/tmp/controller.log
        - name: LOG_ERROR_OUTPUT_PATHS
          value: stderr

I was able to find logs being written to both stdout and /tmp/controller.log. I also noticed that in your karpenter-values.txt, log path is mapped to: /var/karpenter/controller.log and not /var/controller.log. Could this be causing you miss out on those logs?

I will anyways validate this behavior with 1.0.6 as to be sure but please do check if in case you are looking for logs at a different location in place of configured one.

@vijayck2020
Copy link
Author

Thanks for the update @liafizan
I reverted to the older setup of using the config-logging configmap and now the karpenter is successfully writing the logs.

@vijayck2020
Copy link
Author

Now I will introduce some log rotation mechanism to make sure the logs does not eat up my whole storage

@liafizan
Copy link
Contributor

Thanks for the update @liafizan I reverted to the older setup of using the config-logging configmap and now the karpenter is successfully writing the logs.

are you on v1?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good-first-issue Good for newcomers help-wanted Extra attention is needed triage/needs-investigation Issues that need to be investigated before triaging
Projects
None yet
Development

No branches or pull requests

3 participants