Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot use Airbyte on Kubernetes due to OOM when setting up connector #48816

Open
jomach opened this issue Dec 5, 2024 · 0 comments
Open

Cannot use Airbyte on Kubernetes due to OOM when setting up connector #48816

jomach opened this issue Dec 5, 2024 · 0 comments

Comments

@jomach
Copy link

jomach commented Dec 5, 2024

Helm Chart Version

1.2.0

What step the error happened?

During the Sync

Relevant information

Hi, I deployed Airbyte on K8 with default values.yaml. Deployment works and I can login. When I try to add a postgres or a databricks destination the step of checking if everything works fails. The problem is that the resources are not respected from the values.yml:

  jobs:
    resources:
      ## Example:
      ## requests:
      ##    memory: 256Mi
      ##    cpu: 250m
      # -- Job resource requests
      requests:
        cpu: 500m
        memory: 1Gi
      limits:
        cpu: 500m
        memory: 1Gi

From what I understood the values of jobs.resources are passed into the vars JOB_MAIN_CONTAINER_MEMORY_REQUEST etc. But these are being ignored by airbyte when it spawns the pods.

Relevant log output

kubectl describe pod gives me: 


   State:          Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Thu, 05 Dec 2024 08:19:47 +0100
      Finished:     Thu, 05 Dec 2024 08:19:50 +0100
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1100m
      memory:  256Mi
    Requests:
      cpu:     10m
      memory:  1Mi
    Environment:
      AIRBYTE_VERSION:                 1.2.0
      AIRBYTE_ROLE:
      DEPLOYMENT_MODE:                 OSS
      USE_RUNTIME_SECRET_PERSISTENCE:  false
      OPERATION_TYPE:                  check
      WORKLOAD_ID:                     072d5540-f236-4294-ba7c-ade8fd918496_e1032a7c-f92d-4460-86a3-164bff86258f_0_check
    Mounts:
      /config from airbyte-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tj4gr (ro)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants