DAOS-19005 test: Fix dfuse daos_build tests by phender · Pull Request #18363 · daos-stack/daos

phender · 2026-05-26T16:20:58Z

Resolve issues installing elftools and hatchling for daos_build tests.

Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: DaosBuild DaosBuildVM

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

Resolve issues installing elftools and hatchling for daos_build tests. Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daltonbohning · 2026-05-26T16:33:31Z

+            f'{sys.executable} -m pip install pip --upgrade',
+            f'{sys.executable} -m pip install -r {build_dir}/requirements-build.txt',


I wonder if something got missed somewhere because above is

f'{sys.executable} -m venv {mount_dir}/venv'

And even higher is

remote_env['VIRTUAL_ENV'] = os.path.join(mount_dir, 'venv')

But it doesn't seem that we ever source that virtual env?
Maybe we should remove the venv and VIRTUAL_ENV stuff from this test now and just use sys.executable as you're doing

github-actions · 2026-05-26T16:54:34Z

Ticket title is 'dfuse/daos_build_vm.py:DaosBuildVM.test_dfuse_daos_build_* - elftools / hatchling issues'
Status is 'In Progress'
Labels: 'ci_master_weekly,triaged,weekly_test'
https://daosio.atlassian.net/browse/DAOS-19005

daosbuild3 · 2026-05-26T18:43:48Z

Test stage Test RPMs on Leap 15.5 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/2/execution/node/591/log

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-27T00:07:44Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/3/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-27T17:36:05Z

Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/5/execution/node/635/log

daosbuild3 · 2026-05-29T02:09:53Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/6/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-29T22:26:33Z

Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/9/execution/node/573/log

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T14:01:16Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/10/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T14:51:51Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/11/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T17:15:40Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/12/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T19:07:26Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/13/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T22:11:26Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/14/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T23:33:07Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/15/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T06:52:55Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/16/testReport/

daltonbohning · 2026-06-02T12:39:28Z

+    echo "  -p, --python_cmd <val>    Python command to use (default: python3)"
+    echo "  -v, --python_venv <val>   Path to Python virtual environment (default: /tmp/daos_build/venv)"
+    echo "  -b, --build_dir <val>     Directory to clone and build DAOS (default: /tmp/daos_build/daos)"
+    echo "  -g, --git_checkout <val>  Git branch or commit to checkout (default: origin/master)"
+    echo "  -d, --distro <val>        Linux distribution for installing dependencies (default: el9)"
+    echo "  -j, --build_jobs <val>    Number of parallel jobs for building DAOS (default: 30)"
+    echo "  -f, --filesystem_test     Whether to run filesystem tests (default: false)"
+    echo "  -r, --rebuild             Whether to skip setup of build and venv directories (default: false)"
+    echo "  -h, --help                Show this help message and exit"


IMO it's better to use dashses instead of underscores in argument names because that is much more common practice

Fixed in 3d68316

daltonbohning · 2026-06-02T12:42:08Z

+
+# Create a Python virtual environment and install python build dependencies
+if [ "${rebuild}" = "false" ]; then
+    run_cmd 1m "rm -rf ${python_venv}" || exit $?


FYI you don't need the $? because that is piped through. So you could simplify to

Suggested change

run_cmd 1m "rm -rf ${python_venv}" || exit $?

run_cmd 1m "rm -rf ${python_venv}" || exit

Fixed in c1933fe

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T15:41:40Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/17/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T18:33:52Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/19/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T21:26:07Z

Test stage Functional on SLES 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/20/execution/node/623/log

daosbuild3 · 2026-06-02T21:41:57Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/20/testReport/

daosbuild3 · 2026-06-02T21:48:29Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/20/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T23:35:06Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/21/testReport/

daosbuild3 · 2026-06-03T01:55:01Z

Test stage Functional on Leap 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/21/testReport/

daosbuild3 · 2026-06-03T02:09:16Z

Test stage Functional on SLES 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/21/testReport/

daosbuild3 · 2026-06-03T03:18:06Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/21/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-03T14:32:54Z

Test stage Functional on SLES 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/23/testReport/

daosbuild3 · 2026-06-03T14:51:26Z

Test stage Functional on Leap 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/23/testReport/

daosbuild3 · 2026-06-03T14:57:01Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/23/testReport/

daosbuild3 · 2026-06-03T15:26:54Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/23/testReport/

phender added 2 commits May 26, 2026 12:16

DAOS-19005 test: Fix dfuse daos_build tests

99a763e

Resolve issues installing elftools and hatchling for daos_build tests. Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Merge branch 'master' into hendersp/DAOS-19005

efadc77

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daltonbohning reviewed May 26, 2026

View reviewed changes

Add sourcing the virtual environment.

768faf8

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 2 commits May 27, 2026 12:05

Merge branch 'master' into hendersp/DAOS-19005

09b88c6

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Fix command join.

2bf2f64

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 4 commits May 29, 2026 15:30

Merge branch 'master' into hendersp/DAOS-19005

6baaec0

Run a build commands in a remote script

deec53b

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Cleanup

55aa48a

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Make new file executable

fea782c

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Fix joining strings

2baa9c6

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

FIx path to daos_build.sh

3f8eec0

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Update

dbc7808

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Make daos_build.sh executable

aaf5698

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Updated daos_build.sh

ac704bb

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Updates

0cd12fd

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Bump timeouts

6f14600

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daltonbohning reviewed Jun 2, 2026

View reviewed changes

Sudo install meson

3d68316

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Apply feedback and increase git clone timeout

c1933fe

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Eensure venv is sourced for timeout commands

255ea82

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Remove timing commands in daos_build.sh

6488080

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 2 commits June 3, 2026 09:01

Fix mount_dir use.

962057e

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Exit if which debug command fails

21993f9

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

		f'{sys.executable} -m pip install pip --upgrade',
		f'{sys.executable} -m pip install -r {build_dir}/requirements-build.txt',

	run_cmd 1m "rm -rf ${python_venv}" \|\| exit $?
	run_cmd 1m "rm -rf ${python_venv}" \|\| exit

Conversation

phender commented May 26, 2026

Steps for the author:

After all prior steps are complete:

Uh oh!

daltonbohning May 26, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daosbuild3 commented May 26, 2026

Uh oh!

daosbuild3 commented May 27, 2026

Uh oh!

daosbuild3 commented May 27, 2026

Uh oh!

daosbuild3 commented May 29, 2026

Uh oh!

daosbuild3 commented May 29, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daltonbohning Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

phender Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

daltonbohning Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

phender Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

daosbuild3 commented Jun 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented May 26, 2026 •

edited

Loading