Repository navigation
[BUG] Log Links should work with CloudWatch FluentD Out Of the Box #2635
Description
Describe the bug
As it stands, users wishing to enable CloudWatch logs will need to go through the AWS Process to do so here and then manually modify the fluent-bit-config configmap to alter the generated log stream to match what Flyte expects.
kubectl edit cm -n amazon-cloudwatch fluent-bit-config
[OUTPUT]
Name cloudwatch_logs
Match application.*
region ${AWS_REGION}
log_group_name /aws/containerinsights/${CLUSTER_NAME}/application
log_stream_prefix var.
auto_create_group true
extra_user_agent container-insights
And then use this configmap for flyte:
task_logs.yaml: |
plugins:
logs:
cloudwatch-template-uri: 'https://{vars.region}.console.aws.amazon.com/cloudwatch/home?region={vars.region}#logsV2:log-groups/log-group/$252Faws$252Fcontainerinsights$252F<log group name>$252Fapplication$3FlogStreamNameFilter$3Dvar.application.var.log.containers.{{ .podName }}_{{ .namespace }}_{{ .containerName }}'
We should change the default template defined here to be: https://console.aws.amazon.com/cloudwatch/home?region=%s#logsV2:log-groups/log-group/%s$3FlogStreamNameFilter=var.log.containers.{{ .podName }}_{{ .namespace }}_{{ .containerName }}
Expected behavior
- Follow AWS Guide to deploy FluentD: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html
- Enable CloudWatch logs in Flyte:
task_logs.yaml: | plugins: logs: cloudwatch-enabled: true cloudwatch-log-group: 'bv-ml-pipelines' cloudwatch-region: 'us-east-1' kubernetes-enabled: false - SUCCESS ✔️
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
- Yes
Have you read the Code of Conduct?
- Yes
Activity
Azanul commentedon Sep 30, 2022
I'd like to work on this.
Azanul commentedon Sep 30, 2022
#take
Azanul commentedon Sep 30, 2022
#self-assign
Azanul commentedon Oct 2, 2022
@samhita-alla here
Azanul commentedon Oct 15, 2022
#self-assign
Smartmind12 commentedon Oct 22, 2022
@samhita-alla Kindly review the PR and let me know if any changes are required...Thanks!
andrusha commentedon Nov 25, 2022
This assumes that the cluster is going to be deployed with FluentBit by-default, while FluentD is valid and recommended option, to which the older configuration is pointing to. This change will not be applicable for the all users.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html
EngHabu commentedon Nov 26, 2022
@andrusha do you mind elaborating on this? Are you commenting on the discussion we are having on the PR? flyteorg/flyteplugins#293 (comment)
andrusha commentedon Nov 26, 2022
EngHabu commentedon Nov 26, 2022
Aha, I understand, I agree changing the default can break other people, do you know for a fact that FlutentBit in compatibility mode works out of the box? I vaguely recall hostname was still a requirement though, no?
Perhaps a better approach is to:
github-actions commentedon Jun 19, 2024
Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable.
Thank you for your contribution and understanding! 🙏
github-actions commentedon May 14, 2025
Hello 👋, this issue has been inactive for over 90 days and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏