Add support for providing userdata to system VMs

[vishesh92]

Description

This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands.

We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script.

Added global settings:
systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature.
consoleproxy.userdata - UUID of the User data to be used for Console Proxy
secstorage.userdata - UUID of the User data to be used for Secondary Storage VM
router.userdata - UUID of the User data to be used for Virtual Routers

Docs PR: apache/cloudstack-documentation#567

Generated summary

This pull request introduces support for providing user data to system VMs (Console Proxy VM, Secondary Storage VM, and Virtual Router) via global settings, enabling cloud-init configuration through user-supplied data. The changes include new configuration keys, logic to inject user data into VM boot arguments (encoded in base64), and updates to the system VM template to better support cloud-init.

System VM user data support:

• Added systemvm.userdata.enabled config key to globally enable user data for system VMs, and made it available in the VirtualMachineManager and related implementations. [1] [2]
• Introduced new config keys for user data:
  • consoleproxy.userdata for Console Proxy VMs
  • secstorage.userdata for Secondary Storage VMs
  • router.userdata for Virtual Routers
• Updated getConfigKeys() methods in relevant managers to include the new config keys. [1] [2] [3]

User data injection logic:

• In each system VM manager (ConsoleProxyManagerImpl, SecondaryStorageManagerImpl, VirtualNetworkApplianceManagerImpl), added logic to:
  • Check if user data is enabled for the zone and, if so, retrieve and base64-encode the configured user data string.
  • Inject the encoded user data into the VM's boot arguments as userdata=.... [1] [2] [3]

System VM template / cloud-init configuration:

• Updated configure_systemvm_services.sh to change the default cloud-init configuration, switching the data source to NoCloud and disabling automatic cloud-init services to allow manual control by CloudStack.

These changes collectively enable administrators to inject custom cloud-init user data into system VMs via global settings, improving flexibility and automation for system VM initialization.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov]

Codecov Report

❌ Patch coverage is 50.00000% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.40%. Comparing base (69998f7) to head (19af2a7).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...pache/cloudstack/userdata/UserDataManagerImpl.java	0.00%	13 Missing ⚠️
.../network/lb/InternalLoadBalancerVMManagerImpl.java	0.00%	10 Missing ⚠️
.../secondarystorage/SecondaryStorageManagerImpl.java	0.00%	10 Missing ⚠️
...oud/network/lb/ElasticLoadBalancerManagerImpl.java	0.00%	9 Missing ⚠️
...om/cloud/consoleproxy/ConsoleProxyManagerImpl.java	0.00%	9 Missing ⚠️
...ork/router/VirtualNetworkApplianceManagerImpl.java	0.00%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##               main   #11654    +/-   ##
==========================================
  Coverage     17.39%   17.40%            
+ Complexity    15284    15283     -1     
==========================================
  Files          5890     5890            
  Lines        526174   526296   +122     
  Branches      64233    64245    +12     
==========================================
+ Hits          91540    91603    +63     
- Misses       424289   424348    +59     
  Partials      10345    10345            

Flag	Coverage Δ
uitests	3.62% <ø> (ø)
unittests	18.45% <50.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15055

[Copilot]

Pull Request Overview

This PR adds support for providing user data (cloud-init) to system VMs via Zone Scoped global settings, enabling operators to customize Console Proxy VMs, Secondary Storage VMs, and Virtual Routers with monitoring, logging, or custom commands.

• Added new global settings to enable user data feature and configure user data for each system VM type
• Implemented user data injection logic in system VM managers to encode and append user data to boot arguments
• Updated system VM template cloud-init configuration to use NoCloud datasource and disable automatic cloud-init services

Reviewed Changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
engine/api/src/main/java/com/cloud/vm/VirtualMachineManager.java	Added global setting to enable user data for system VMs
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java	Updated config keys array to include the new system VM user data setting
server/src/main/java/com/cloud/consoleproxy/ConsoleProxyManager.java	Added configuration key for console proxy user data
server/src/main/java/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java	Implemented user data injection logic for console proxy VMs
server/src/main/java/com/cloud/network/router/VirtualNetworkApplianceManager.java	Added configuration key for virtual router user data
server/src/main/java/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java	Implemented user data injection logic for virtual router VMs
server/src/main/java/com/cloud/storage/secondary/SecondaryStorageVmManager.java	Added configuration key for secondary storage user data
services/secondary-storage/controller/src/main/java/org/apache/cloudstack/secondarystorage/SecondaryStorageManagerImpl.java	Implemented user data injection logic for secondary storage VMs
tools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh	Updated cloud-init configuration to use NoCloud datasource and disabled automatic services

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 15057

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15064

[harikrishna-patnala]

@vishesh92 is it a good idea to add some validations to these userdata configs, to avoid any kind of security risks ?

[vishesh92]

@vishesh92 is it a good idea to add some validations to these userdata configs, to avoid any kind of security risks ?

IMO, it would be difficult to do that. Technically the user data could be a script which fetches a binary (malicious) from internet and executes it on the system VM and we won't be able to validate this.
But since the global settings are scoped at zone level, only the operators can set the user data.

I have also added a new global setting, systemvm.userdata.enabled, which is set to false by default. Unless this is set to true, the other global settings are ignored.

[weizhouapache]

code lgtm

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[harikrishna-patnala]

code LGTM. good to see initially this is allowed only for default root admins. thanks @vishesh92

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15145

[borisstoyanov]

@blueorangutan test

[blueorangutan]

@borisstoyanov a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests