Skip to content

QoS Control: DiffServ #2290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ctauchen merged 1 commit into from
Sep 22, 2025
Merged

QoS Control: DiffServ #2290

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions calico-cloud/networking/configuring/qos-controls.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@

## Big picture

Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by Calico workloads in order to prevent them from overusing or overloading network resources.
Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by $[prodname] workloads in order to prevent them from overusing or overloading network resources.
Additionally, it allows configuring Differentiated Services (DiffServ) on egress traffic of $[prodname] workloads and hosts.

## Value

With QoS controls, $[prodname] enforces network resource limits (bandwidth, packet rate, connections) for Kubernetes pods, ensuring fair resource allocation and preventing performance degradation for other workloads.
With QoS controls, $[prodname] enforces network resource limits (bandwidth, packet rate, connections) for Kubernetes pods and OpenStack VMs, ensuring fair resource allocation and preventing performance degradation for other workloads.
$[prodname] can also apply DiffServ on egress traffic to allow upstream devices to prioritise forwarding accordingly.

## Limitations

Expand All @@ -26,6 +28,7 @@
1. Bandwidth: limiting the bit rate of traffic into/out of the pod
2. Packet rate: limiting the number of packets per second that can be sent or received by the pod
3. Number of established connections: limiting the absolute number of connections that the pod may initiate or accept
4. Differentiated Services (DiffServ): setting Differentiated Services Code Point (DSCP) on egress traffic leaving clusters or toward hosts

### Annotations

Expand Down Expand Up @@ -58,6 +61,7 @@
- [Configure bandwidth limits](#configure-bandwidth-limits)
- [Configure packet rate limits](#configure-packet-rate-limits)
- [Configure established connection limits](#configure-established-connection-limits)
- [Configure DiffServ](#configure-diffserv)

### Configure bandwidth limits

Expand Down Expand Up @@ -104,3 +108,28 @@
| `qos.projectcalico.org/ingressMaxConnections` | Specifies the maximum number of ingress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |
| `qos.projectcalico.org/egressMaxConnections` | Specifies the maximum number of egress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |

### Configure DiffServ

Use the annotation below to set DSCP on egress traffic toward an endpoint outside cluster or another host in cluster. Unlike, other QoS controls, this annotation can also be set on a [host endpoint](../../reference/host-endpoints).
For example, to set DSCP AF11 on a host endpoint:

```yaml noValidation
apiVersion: projectcalico.org/v3
kind: HostEndpoint
metadata:
name: my-hep
annotations:
qos.projectcalico.org/dscp: "AF11"
labels:
app: my-app
spec:
(...)
```

| Annotation | Description |
|---------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `qos.projectcalico.org/dscp` | Specifies the DSCP value of egress traffic toward an endpoint outside cluser or another host in cluster.<br/>Min value: `0`<br/>Max value: `63`<br/>String values: DF, EF, AF11, AF12, AF13, AF21, AF22, AF23, AF31, AF32, AF33, AF41, AF42, AF43, CS0, CS1, CS2, CS3, CS4, CS5, CS6 |

Check failure on line 131 in calico-cloud/networking/configuring/qos-controls.mdx

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Vale.Spelling] Did you really mean 'cluser'?

Raw Output:
{"message": "[Vale.Spelling] Did you really mean 'cluser'?", "location": {"path": "calico-cloud/networking/configuring/qos-controls.mdx", "range": {"start": {"line": 131, "column": 104}}}, "severity": "ERROR"}

## Additional resources

- [QoS controls on Openstack](../openstack/neutron-api.mdx#qos)

Check failure on line 135 in calico-cloud/networking/configuring/qos-controls.mdx

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Vale.Spelling] Did you really mean 'Openstack'?

Raw Output:
{"message": "[Vale.Spelling] Did you really mean 'Openstack'?", "location": {"path": "calico-cloud/networking/configuring/qos-controls.mdx", "range": {"start": {"line": 135, "column": 20}}}, "severity": "ERROR"}
33 changes: 31 additions & 2 deletions calico-enterprise/networking/configuring/qos-controls.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@

## Big picture

Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by Calico workloads in order to prevent them from overusing or overloading network resources.
Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by $[prodname] workloads in order to prevent them from overusing or overloading network resources.
Additionally, it allows configuring Differentiated Services (DiffServ) on egress traffic of $[prodname] workloads and hosts.

## Value

With QoS controls, $[prodname] enforces network resource limits (bandwidth, packet rate, connections) for Kubernetes pods, ensuring fair resource allocation and preventing performance degradation for other workloads.
With QoS controls, $[prodname] enforces network resource limits (bandwidth, packet rate, connections) for Kubernetes pods and OpenStack VMs, ensuring fair resource allocation and preventing performance degradation for other workloads.
$[prodname] can also apply DiffServ on egress traffic to allow upstream devices to prioritise forwarding accordingly.

## Limitations

Expand All @@ -26,6 +28,7 @@
1. Bandwidth: limiting the bit rate of traffic into/out of the pod
2. Packet rate: limiting the number of packets per second that can be sent or received by the pod
3. Number of established connections: limiting the absolute number of connections that the pod may initiate or accept
4. Differentiated Services (DiffServ): setting Differentiated Services Code Point (DSCP) on egress traffic leaving clusters or toward hosts

### Annotations

Expand Down Expand Up @@ -58,6 +61,7 @@
- [Configure bandwidth limits](#configure-bandwidth-limits)
- [Configure packet rate limits](#configure-packet-rate-limits)
- [Configure established connection limits](#configure-established-connection-limits)
- [Configure DiffServ](#configure-diffserv)

### Configure bandwidth limits

Expand Down Expand Up @@ -104,3 +108,28 @@
| `qos.projectcalico.org/ingressMaxConnections` | Specifies the maximum number of ingress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |
| `qos.projectcalico.org/egressMaxConnections` | Specifies the maximum number of egress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |

### Configure DiffServ

Use the annotation below to set DSCP on egress traffic toward an endpoint outside cluster or another host in cluster. Unlike, other QoS controls, this annotation can also be set on a [host endpoint](../../reference/host-endpoints).
For example, to set DSCP AF11 on a host endpoint:

```yaml noValidation
apiVersion: projectcalico.org/v3
kind: HostEndpoint
metadata:
name: my-hep
annotations:
qos.projectcalico.org/dscp: "AF11"
labels:
app: my-app
spec:
(...)
```

| Annotation | Description |
|---------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `qos.projectcalico.org/dscp` | Specifies the DSCP value of egress traffic toward an endpoint outside cluser or another host in cluster.<br/>Min value: `0`<br/>Max value: `63`<br/>String values: DF, EF, AF11, AF12, AF13, AF21, AF22, AF23, AF31, AF32, AF33, AF41, AF42, AF43, CS0, CS1, CS2, CS3, CS4, CS5, CS6 |

Check failure on line 131 in calico-enterprise/networking/configuring/qos-controls.mdx

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Vale.Spelling] Did you really mean 'cluser'?

Raw Output:
{"message": "[Vale.Spelling] Did you really mean 'cluser'?", "location": {"path": "calico-enterprise/networking/configuring/qos-controls.mdx", "range": {"start": {"line": 131, "column": 104}}}, "severity": "ERROR"}

## Additional resources

- [QoS controls on Openstack](../openstack/neutron-api.mdx#qos)

Check failure on line 135 in calico-enterprise/networking/configuring/qos-controls.mdx

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Vale.Spelling] Did you really mean 'Openstack'?

Raw Output:
{"message": "[Vale.Spelling] Did you really mean 'Openstack'?", "location": {"path": "calico-enterprise/networking/configuring/qos-controls.mdx", "range": {"start": {"line": 135, "column": 20}}}, "severity": "ERROR"}
28 changes: 27 additions & 1 deletion calico/networking/configuring/qos-controls.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@

## Big picture

Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by Calico workloads in order to prevent them from overusing or overloading network resources.
Configure QoS (Quality of Service) Controls to limit ingress and/or egress bandwidth, packet rate and number of connections used by $[prodname] workloads in order to prevent them from overusing or overloading network resources.
Additionally, it allows configuring Differentiated Services (DiffServ) on egress traffic of $[prodname] workloads and hosts.

## Value

With QoS controls, $[prodname] enforces network resource limits (bandwidth, packet rate, connections) for Kubernetes pods and OpenStack VMs, ensuring fair resource allocation and preventing performance degradation for other workloads.
$[prodname] can also apply DiffServ on egress traffic to allow upstream devices to prioritise forwarding accordingly.

## Limitations

Expand All @@ -26,6 +28,7 @@
1. Bandwidth: limiting the bit rate of traffic into/out of the pod
2. Packet rate: limiting the number of packets per second that can be sent or received by the pod
3. Number of established connections: limiting the absolute number of connections that the pod may initiate or accept
4. Differentiated Services (DiffServ): setting Differentiated Services Code Point (DSCP) on egress traffic leaving clusters or toward hosts

### Annotations

Expand Down Expand Up @@ -58,6 +61,7 @@
- [Configure bandwidth limits](#configure-bandwidth-limits)
- [Configure packet rate limits](#configure-packet-rate-limits)
- [Configure established connection limits](#configure-established-connection-limits)
- [Configure DiffServ](#configure-diffserv)

### Configure bandwidth limits

Expand Down Expand Up @@ -104,6 +108,28 @@
| `qos.projectcalico.org/ingressMaxConnections` | Specifies the maximum number of ingress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |
| `qos.projectcalico.org/egressMaxConnections` | Specifies the maximum number of egress connections.<br/>Min value: `1`<br/>Max value: `4294967295` |

### Configure DiffServ

Use the annotation below to set DSCP on egress traffic toward an endpoint outside cluster or another host in cluster. Unlike, other QoS controls, this annotation can also be set on a [host endpoint](../../reference/host-endpoints).
For example, to set DSCP AF11 on a host endpoint:

```yaml noValidation
apiVersion: projectcalico.org/v3
kind: HostEndpoint
metadata:
name: my-hep
annotations:
qos.projectcalico.org/dscp: "AF11"
labels:
app: my-app
spec:
(...)
```

| Annotation | Description |
|---------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `qos.projectcalico.org/dscp` | Specifies the DSCP value of egress traffic toward an endpoint outside cluser or another host in cluster.<br/>Min value: `0`<br/>Max value: `63`<br/>String values: DF, EF, AF11, AF12, AF13, AF21, AF22, AF23, AF31, AF32, AF33, AF41, AF42, AF43, CS0, CS1, CS2, CS3, CS4, CS5, CS6 |

Check failure on line 131 in calico/networking/configuring/qos-controls.mdx

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Vale.Spelling] Did you really mean 'cluser'?

Raw Output:
{"message": "[Vale.Spelling] Did you really mean 'cluser'?", "location": {"path": "calico/networking/configuring/qos-controls.mdx", "range": {"start": {"line": 131, "column": 104}}}, "severity": "ERROR"}

## Additional resources

- [QoS controls on Openstack](../openstack/neutron-api.mdx#qos)