Document how to change the MTU

w13915984028 · w13915984028 · commit 789d17b9d563 · 2024-09-11T11:59:14.000+02:00
Signed-off-by: Jian Wang &lt;jian.wang@suse.com&gt;
diff --git a/docs/advanced/storagenetwork.md b/docs/advanced/storagenetwork.md
@@ -140,6 +140,10 @@ Because of the design, Harvester will treat extra and insignificant characters i
 
 :::
 
+#### Change the MTU of the Storage Network
+
+When you plan to change the MTU of the Storage Network, please follow the guide [Change the MTU of Network Config which has Storage Network Attached](../networking/clusternetwork.md#change-the-mtu-of-network-config-which-has-storage-network-attached).
+
 ### After Applying Harvester Storage Network Setting
 
 After applying Harvester's Storage Network setting, Harvester will stop all pods that are related to Longhorn volumes. Currently, Harvester has some pods listed below that will be stopped during setting.
diff --git a/docs/networking/clusternetwork.md b/docs/networking/clusternetwork.md
@@ -93,9 +93,11 @@ The [witness node](../advanced/witness.md) is generally not involved in the cust
 
 :::
 
-#### How to create a new cluster network
+## Configurations
 
-1. To create a cluster network, go to the **Networks > ClusterNetworks/Configs** page and click the **Create** button. You only need to specify the name.
+### How to Create a new Cluster Network
+
+1. To create a new cluster network, go to the **Networks > ClusterNetworks/Configs** page and click the **Create** button. You only need to specify the name.
 
    ![](/img/v1.2/networking/create-clusternetwork.png)
 
@@ -114,7 +116,7 @@ The method **Select all nodes** works only when all nodes use the exact same ded
 :::
 
 4. Click the **Uplink** tab to add the NICs, and configure the bond options and link attributes. The bond mode defaults to `active-backup`.
-    
+
    ![](/img/v1.2/networking/config-uplink.png)
 
 :::note
@@ -131,3 +133,110 @@ Starting with Harvester v1.1.2, Harvester supports updating network configs. Mak
 To simplify cluster maintenance, create one network configuration for each node or group of nodes. Without dedicated network configurations, certain maintenance tasks (for example, replacing old NICs with NICs in different slots) will require you to stop and/or migrate the affected VMs before updating the network configuration.
 
 :::
+
+### How to Change the Network Config
+
+Changes on the existing `Network Config` may affect both the Harvester VMs/workloads and the external devices/systems like Switches/Routers. For more information, please see [Network Topology](./deep-dive.md#network-topology).
+
+#### Change the MTU of Network Config which has no Storage Network attached
+
+You may plan to change the `MTU` of an existing `Cluster Network`. And the [Storage Network](../advanced/storagenetwork.md#harvester-storage-network-setting) is not enabled or not attached to this `Cluster Network`.
+
+The `MTU` on each `Network Config` of an existing custom `Cluster Network` is strictly to be identical. There are many restrictions to change the `MTU`, the following steps should be followed:
+
+1. Stop all the VMs which are attached to the target `Cluster Network`, this can be checked via the [VM Network](./harvester-network.md#create-a-vm-network) and [VM attached Secondary Network](../vm/create-vm.md#secondary-network). When any of the VMs is still running, Harvester will refuse the change.
+
+2. Check the number of `Network Config` of the target `Cluster Network`, if the number is greater than one, then repeat the operations below until there is only one `Network Config` left:
+
+    1. Record the `Node Selector` of a `Network Config`;
+
+    1. Remove this `Network Config`;
+
+3. Change the `MTU` of the last `Network Config`.
+
+4. Check the `MTU` on the selected Harvester nodes via the Linux `ip link` command, the related `*-br` device should be `UP` and with the new `MTU`.
+
+```
+Harvester node $ ip link
+
+                                                  |new MTU|              |state UP|
+3: cn-data-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
+    link/ether 52:54:00:6e:5c:2a brd ff:ff:ff:ff:ff:ff
+```
+
+5. Test the new MTU on Harvester nodes via command like `ping` to another Harvester node (with the new MTU) or an external device.
+
+```
+Harvester node $ ping 8.8.8.8 -s 8800
+
+PING 8.8.8.8 (8.8.8.8) 8800(8828) bytes of data
+
+# the `-s` specify the ping packet size, which can test if the new MTU really works
+
+```
+
+6. Add those `Network Config` which were removed on Step 2, each should set the `MTU` to the new value. And run step 4, 5 to test and verify the new `MTU`.
+
+7. Start the VMs mentioned in step 1.
+
+:::note
+
+- The `MTU` affects both of the Harvester nodes and the infrastructure networking devices like Switches and Routers, the careful planning and testing are required to sure the new `MTU`. For more information, please see [Network Topology](./deep-dive.md#network-topology).
+
+- This method does not work on the built-in `mgmt Cluster Network`.
+
+:::
+
+#### Change the MTU of Network Config which has Storage Network Attached
+
+You may plan to change the `MTU` of an existing `Cluster Network`. And the [Storage Network](../advanced/storagenetwork.md#harvester-storage-network-setting) is enabled or attached to this `Cluster Network`. The `Storage Network` is dedicatedly used by `Longhorn`, the default CSI driver of Harvester cluster. Because the `Longhorn` is responsible for at least the [root disk](../vm/create-vm.md#volumes) of all VMs, this change will affect all the VMs.
+
+The `MTU` on each `Network Config` of an existing custom `Cluster Network` is strictly to be identical. There are many restrictions to change the `MTU`, the following steps should be followed:
+
+1. Stop all the VMs.
+
+2. Disable the Harvester [Storage Network](../advanced/storagenetwork.md#harvester-storage-network-setting). Wait and [Verify Configuration is Completed](../advanced/storagenetwork.md#verify-configuration-is-completed).
+
+3. Check the number of `Network Config` of the target `Cluster Network`, if the number is greater than one, then repeat the operations below until there is only one `Network Config` left:
+
+    1. Record the `Node Selector` of a `Network Config`;
+
+    1. Remove this `Network Config`;
+
+4. Change the `MTU` of the last `Network Config`.
+
+5. Check the `MTU` on the selected Harvester nodes via the Linux `ip link` command, the related `*-br` device should be `UP` and with the new `MTU`.
+
+```
+Harvester node $ ip link
+
+                                                  |new MTU|              |state UP|
+
+3: cn-data-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
+    link/ether 52:54:00:6e:5c:2a brd ff:ff:ff:ff:ff:ff
+```
+
+6. Test the new MTU on Harvester nodes via command like `ping` to another Harvester node (with the new MTU) or an external device.
+
+```
+Harvester node $ ping 8.8.8.8 -s 8800
+
+PING 8.8.8.8 (8.8.8.8) 8800(8828) bytes of data
+
+# the `-s` specify the ping packet size, which can test if the new MTU really works
+
+```
+
+7. Add those `Network Config` which were removed on Step 3, each should set the `MTU` to the new value. And run step 5, 6 to test and verify the new `MTU`.
+
+8. Enable and set the Harvester [Storage Network](../advanced/storagenetwork.md#harvester-storage-network-setting), note the [Prerequisites](../advanced/storagenetwork.md#prerequisites) are met in the above steps. Wait and [Verify Configuration is Completed](../advanced/storagenetwork.md#verify-configuration-is-completed).
+
+9. Start the VMs mentioned in step 1.
+
+:::note
+
+- The `MTU` affects both of the Harvester nodes and the infrastructure networking devices like Switches and Routers, the careful planning and testing are required to sure the new `MTU`. For more information, please see [Network Topology](./deep-dive.md#network-topology).
+
+- This method does not work on the built-in `mgmt Cluster Network`.
+
+:::
diff --git a/docs/networking/deep-dive.md b/docs/networking/deep-dive.md
@@ -12,6 +12,8 @@ keywords:
   <link rel="canonical" href="https://docs.harvesterhci.io/v1.1/networking/deep-dive"/>
 </head>
 
+## Network Topology
+
 The network topology below reveals how we implement the Harvester network.
 
 ![](/img/v1.2/networking/topology.png)
diff --git a/docs/networking/harvester-network.md b/docs/networking/harvester-network.md
@@ -69,6 +69,14 @@ The [Harvester network-controller](https://github.com/harvester/harvester-networ
 
     ![](/img/v1.2/networking/create-vlan-network.png)
 
+:::note
+
+The `VM Network` inheriates the `MTU` from the `Network Config` of the `Cluster Network`, this makes sure the VMs will get the best match and best performance from the hardware. It is not allowed to set a different `MTU` on the `VM Network`.
+
+Follow the guide [Change the MTU of Network Config which has Storage Network Attached](./clusternetwork.md#change-the-mtu-of-network-config-which-has-storage-network-attached) or [Change the MTU of Network Config which has no Storage Network Attached](./clusternetwork.md#change-the-mtu-of-network-config-which-has-no-storage-network-attached) to set a different `MTU`, then the `VM Network` will have the new `MTU` automatically.
+
+:::
+
 1. On the Route tab, select an option and then specify the related IPv4 addresses.
 
     - Auto(DHCP): The Harvester network controller retrieves the CIDR and gateway addresses from the DHCP server. You can specify the DHCP server address. 
diff --git a/docs/vm/create-vm.md b/docs/vm/create-vm.md
@@ -37,7 +37,7 @@ Please refer to [this page](./create-windows-vm.md) for creating Windows virtual
 1. (Optional) VM template is optional, you can choose `iso-image`, `raw-image` or `windows-iso-image` template to speed up your VM instance creation.
 1. Configure the virtual machine's CPU and memory (see [overcommit settings](../advanced/settings.md#overcommit-config) if you want to over-provision).
 1. Select SSH keys or upload new keys.
-1. Select a custom VM image on the **Volumes** tab. The default disk will be the root disk. You can add more disks to the VM.
+1. Select a custom VM image on the **Volumes** tab. The default disk will be the `root disk`. You can add more disks to the VM.
 1. To configure networks, go to the **Networks** tab. 
     1. The **Management Network** is added by default, you can remove it if the VLAN network is configured.
     1. You can also add additional networks to the VMs using VLAN networks. You may configure the VLAN networks on **Advanced > Networks** first.
@@ -142,7 +142,7 @@ resource "harvester_virtualmachine" "opensuse154" {
 
 ## Volumes
 
-You can add one or more additional volumes via the `Volumes` tab, by default the first disk will be the root disk, you can change the boot order by dragging and dropping volumes, or using the arrow buttons.
+You can add one or more additional volumes via the `Volumes` tab, by default the first disk will be the `root disk`, you can change the boot order by dragging and dropping volumes, or using the arrow buttons.
 
 A disk can be made accessible via the following types:
 
@@ -167,7 +167,7 @@ By default, Harvester sets the volume size to either 10 GiB or the virtual size
 
 A container disk is an ephemeral storage volume that can be assigned to any number of VMs and provides the ability to store and distribute VM disks in the container image registry. A container disk is:
 - An ideal tool if you want to replicate a large number of VM workloads or inject machine drivers that do not require persistent data. Ephemeral volumes are designed for VMs that need more storage but don't care whether that data is stored persistently across VM restarts or only expect some read-only input data to be present in files, like configuration data or secret keys.
-- Not a good solution for any workload that requires persistent root disks across VM restarts.
+- Not a good solution for any workload that requires persistent `root disks` across VM restarts.
 
 A container disk is added when creating a VM by providing a Docker image. When creating a VM, follow these steps:
 
