Skip to content

Commit cbabbe4

Browse files
committed
set API workers to a default fo 1
1 parent 8607318 commit cbabbe4

File tree

3 files changed

+14
-8
lines changed

3 files changed

+14
-8
lines changed

docs/modules/airflow/pages/troubleshooting/index.adoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ See e.g. https://github.com/minio/minio/issues/20845[this MinIO issue] for detai
2929
== Setting API Workers
3030

3131
In Airflow the webserver (called the API Server in Airflow 3.x+) can use multiple workers.
32-
This is determined by the environment variable `+AIRFLOW__API__WORKERS+` and is set by default to `4`.
33-
For most cases the default should work without problem, but if you run into performance issues and would like to add more workers, you can either modulate multiple worker processes at the level of webserver, keeping the default value for each one:
32+
This is determined by the environment variable `+AIRFLOW__API__WORKERS+` and is set by default to `1`.
33+
For most cases this should work without problem, but if you run into performance issues and would like to add more workers, you can either modulate multiple worker processes at the level of webserver (keeping the default value for each instance):
3434

3535
[source,yaml]
3636
----
@@ -46,7 +46,7 @@ or change the environment variable using `envOverrides`:
4646
----
4747
webservers:
4848
envOverrides:
49-
AIRFLOW__API__WORKERS: "6" # something other than the default
49+
AIRFLOW__API__WORKERS: "4" # something other than the default
5050
----
5151

5252
TIP: Our strong recommendation is to increase the webserver replicas, with each webserver running a single worker, as this removes the risk of running into timeouts or memory issues.

rust/operator-binary/src/env_vars.rs

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -505,6 +505,17 @@ fn add_version_specific_env_vars(
505505
JWT_SECRET_SECRET_KEY,
506506
),
507507
);
508+
// The airflow default for this is 4, but we document a recommended
509+
// value of 1 with the option of either increasing it via overrides
510+
// or of adding webserver instances.
511+
env.insert(
512+
"AIRFLOW__API__WORKERS".into(),
513+
EnvVar {
514+
name: "AIRFLOW__API__WORKERS".into(),
515+
value: Some("1".into()),
516+
..Default::default()
517+
},
518+
);
508519
if airflow_role == &AirflowRole::Webserver {
509520
// Sometimes a race condition can arise when both scheduler and
510521
// api-server are updating the DB, which adds overhead (conflicts

tests/templates/kuttl/logging/41-install-airflow-cluster.yaml.j2

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -84,11 +84,6 @@ spec:
8484
max: 2000m
8585
memory:
8686
limit: 3Gi
87-
envOverrides:
88-
# logging tests use two webservers and if two tests should run in
89-
# parallel then the CPU usage can be high if the default number of
90-
# workers (4) is used
91-
AIRFLOW__API__WORKERS: "1"
9287
roleGroups:
9388
automatic-log-config:
9489
replicas: 1

0 commit comments

Comments
 (0)