Manifest entry in launcher not propagating to tests

[frenchwr]

Bug Description

I have the following launcher script with a manifest for has_npu = true:

#!/usr/bin/env checkbox.checkbox-cli
[manifest]
com.canonical.certification::has_npu = true

[launcher]
launcher_version = 1
app_id = com.canonical.certification::npu-test
stock_reports = text, submission_files

[test plan]
unit = com.canonical.certification::npu-with-known-failures
forced = yes

[test selection]
forced = yes

[ui]
type = silent
auto_retry = no

Based on docs I expected this to be sufficient for running my tests needing this manifest, but the initial run of the launcher is instead resulting in:

$ ./checkbox-launcher 
We are deprecating the classic frontend Checkbox snaps
Consider switching to the strict frontend

$PROVIDERPATH is defined, so following provider sources are ignored ['/home/ubuntu/.local/share/plainbox-providers-1', '/var/tmp/checkbox-providers-develop'] 
Preparing...
==============[ Bootstrap com.canonical.plainbox::manifest (1/2) ]==============
WARNING:checkbox_ng.user_utils:Using `ubuntu` user
======[ Bootstrap com.canonical.certification::npu_gtest_resource (2/2) ]=======
Saving manifest to /var/tmp/checkbox-ng/machine-manifest.json
Reports will be saved to: /home/ubuntu/.local/share/checkbox-ng
==============[ Running job 1 / 3. Estimated time left: 0:00:05 ]===============
-----[ Check that user has read and write access to NPU char device node ]------
ID: com.canonical.certification::kmd/char_device_permissions
Category: com.canonical.certification::npu_kmd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 2 / 3. Estimated time left: 0:00:03 ]===============
-------------------------[ Check NPU firmware version ]-------------------------
ID: com.canonical.certification::kmd/firmware_version
Category: com.canonical.certification::npu_kmd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 3 / 3. Estimated time left: 0:00:01 ]===============
---------------[ Check if we can use npu-umd-test from the snap ]---------------
ID: com.canonical.certification::umd/detect
Category: com.canonical.certification::npu_umd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
 - required dependency 'com.canonical.certification::kmd/char_device_permissions' has failed
Outcome: job cannot be started
 ☑ : Hardware Manifest
 ☐ : Check that user has read and write access to NPU char device node
 ☐ : Check NPU firmware version
 ☐ : Check if we can use npu-umd-test from the snap

Subsequent runs (when /var/tmp/checkbox-ng/machine-manifest.json is present) run fine.

Cert-blocker Test Case

• cert-blocker

To Reproduce

# also tested on latest/beta, I need at least beta for the npu test support since it was added recently
sudo snap install checkbox24 --channel latest/edge
# also tested on latest/stable
sudo snap install checkbox --channel latest/edge --classic
# optional
sudo snap install intel-npu-driver --channel latest/edge

2. Create launcher

#!/usr/bin/env checkbox.checkbox-cli
[manifest]
com.canonical.certification::has_npu = true

[launcher]
launcher_version = 1
app_id = com.canonical.certification::npu-test
stock_reports = text, submission_files

[test plan]
unit = com.canonical.certification::npu-with-known-failures
forced = yes

[test selection]
forced = yes

[ui]
type = silent
auto_retry = no

3. Run launcher

chmod +x checkbox-launcher
./checkbox-launcher

Expected Result

The tests requiring the has_npu=true manifest run successfully the first time you invoke the launcher (without adding /var/tmp/checkbox-ng/machine-manifest.json).

Actual Result

The tests do not run because checkbox does not think the has_npu=true manifest is set when it's not defined in /var/tmp/checkbox-ng/machine-manifest.json.

Environment

• Ubuntu 24.04
• checkbox24 snap version 7.1.0-dev96 (rev 1688)
• checkbox snap version 7.1.0-dev96 (rev 22669)

Relevant log output

$ ./checkbox-launcher 
We are deprecating the classic frontend Checkbox snaps
Consider switching to the strict frontend

$PROVIDERPATH is defined, so following provider sources are ignored ['/home/ubuntu/.local/share/plainbox-providers-1', '/var/tmp/checkbox-providers-develop'] 
Preparing...
==============[ Bootstrap com.canonical.plainbox::manifest (1/2) ]==============
WARNING:checkbox_ng.user_utils:Using `ubuntu` user
======[ Bootstrap com.canonical.certification::npu_gtest_resource (2/2) ]=======
Saving manifest to /var/tmp/checkbox-ng/machine-manifest.json
Reports will be saved to: /home/ubuntu/.local/share/checkbox-ng
==============[ Running job 1 / 3. Estimated time left: 0:00:05 ]===============
-----[ Check that user has read and write access to NPU char device node ]------
ID: com.canonical.certification::kmd/char_device_permissions
Category: com.canonical.certification::npu_kmd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 2 / 3. Estimated time left: 0:00:03 ]===============
-------------------------[ Check NPU firmware version ]-------------------------
ID: com.canonical.certification::kmd/firmware_version
Category: com.canonical.certification::npu_kmd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 3 / 3. Estimated time left: 0:00:01 ]===============
---------------[ Check if we can use npu-umd-test from the snap ]---------------
ID: com.canonical.certification::umd/detect
Category: com.canonical.certification::npu_umd
Job cannot be started because:
 - resource expression "manifest.has_npu == 'True'" evaluates to false
 - required dependency 'com.canonical.certification::kmd/char_device_permissions' has failed
Outcome: job cannot be started
 ☑ : Hardware Manifest
 ☐ : Check that user has read and write access to NPU char device node
 ☐ : Check NPU firmware version
 ☐ : Check if we can use npu-umd-test from the snap

Additional context

No response

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-2221.

This message was autogenerated