DAOS-17519 test: Automate dlck testing (basic/fault_injection) by rpadma2 · Pull Request #17307 · daos-stack/daos

rpadma2 · 2025-12-23T18:53:19Z

Skip-func-hw-test-medium: false
Skip-func-hw-test-medium-md-on-ssd: false
Skip-unit-test: true
Skip-fault-injection-test: true
Skip-nlt: true
Test-tag: DlckBasicFaultTest DlckBasicTest

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

github-actions · 2025-12-23T18:53:37Z

Ticket title is 'Automation - Basic dlck test: scan the DAOS system by running the dlck tool.'
Status is 'In Progress'
Labels: 'test_2.8,testp2'
https://daosio.atlassian.net/browse/DAOS-17519

Test-tag: DlckBasicTest DlckBasicFaultTest Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest

daosbuild3 · 2026-01-08T03:54:18Z

Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/3/execution/node/932/log

daosbuild3 · 2026-01-08T04:09:12Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/3/testReport/

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-03-12T21:34:15Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/4/execution/node/966/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-03-31T22:08:14Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/5/execution/node/988/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-05-23T06:32:25Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/6/execution/node/867/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-05-23T19:58:48Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/7/execution/node/866/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-12T19:16:38Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/8/execution/node/1073/log

daosbuild3 · 2026-08-12T19:17:57Z

Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/8/execution/node/1083/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-13T00:15:36Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/9/execution/node/1072/log

daosbuild3 · 2026-08-13T00:27:22Z

Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/9/execution/node/1082/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daltonbohning · 2026-08-13T16:22:45Z

+            timeout (int, optional): Command timeout (sec) used in run. Defaults to
+                None.
+            sudo (bool, optional): Whether to run dlck with sudo. Defaults to True.
+            env_str (str, optional): Environment variable string to prepend to command.


I think this class should instead inherit from ExecutableCommand and then there is no need to pass env_str because there is already a dictionary of ExecutableCommand.env

Sure. Will take a look at it. Thanks.

daosbuild3 · 2026-08-13T20:15:02Z

Test stage Functional Hardware Medium completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/11/testReport/

daosbuild3 · 2026-08-13T20:46:16Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/11/execution/node/1120/log

shimizukko

You might want to add more log_step where you operate the system (running dmg, dlck, or other Linux commands).

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-17T22:18:10Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/12/execution/node/1071/log

daosbuild3 · 2026-08-17T22:19:06Z

Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/12/execution/node/1081/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-18T02:41:20Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/13/testReport/

Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-19T08:07:21Z

Test stage Functional Hardware Medium completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/14/testReport/

daosbuild3 · 2026-08-19T08:40:01Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17307/14/execution/node/1118/log

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-20T00:14:39Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/15/testReport/

daosbuild3 · 2026-08-20T01:14:57Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/16/testReport/

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-20T17:39:07Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/17/testReport/

Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

daosbuild3 · 2026-08-21T11:02:07Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17307/20/testReport/

DAOS-17519 test: Automate dlck testing (basic/fault_injection)

e44ee36

Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

DAOS-17519 test: Fix linting issues.

1dfe0ef

Test-tag: DlckBasicTest DlckBasicFaultTest Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

janekmi requested changes Dec 29, 2025

View reviewed changes

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

a376c98

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest

rpadma2 added 2 commits March 12, 2026 11:26

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

1f16a51

DAOS-17519 test: Some of the review updates.

aa8ede9

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 3 commits March 31, 2026 10:49

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

5213ba6

DAOS-17519 test: Fix Linting issues.

0cea354

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 2 commits May 22, 2026 13:18

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

593f473

DAOS-17519 test: Update the control_meta_dir.

e4508aa

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 2 commits August 12, 2026 14:18

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

2b4bc8a

rpadma2 requested a review from janekmi August 12, 2026 18:40

rpadma2 added 2 commits August 12, 2026 15:25

DAOS-17519 test: Fix more pylint issues.

2191472

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

DAOS-17519 test: Update test timeouts.

fb11310

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 2 commits August 12, 2026 21:55

DAOS-17519 test: add_pool needed parameter.

7284649

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

DAOS-17519 test: Fix flake8 newline issue.

3b870dc

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 requested review from daltonbohning, phender and shimizukko August 13, 2026 15:30

daltonbohning reviewed Aug 13, 2026

View reviewed changes

shimizukko reviewed Aug 13, 2026

View reviewed changes

Comment thread src/tests/ftest/recovery/dlck_basic_fault.yaml

shimizukko previously approved these changes Aug 13, 2026

View reviewed changes

shimizukko reviewed Aug 13, 2026

View reviewed changes

Comment thread src/tests/ftest/util/dlck_utils.py Outdated

rpadma2 added 2 commits August 17, 2026 14:57

DAOS-17519 test: Merge branch 'master' into rpadma2/daos_17519

3c8a19a

DAOS-17519 test: Python script review updates.

68b4f6b

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 dismissed shimizukko’s stale review via 68b4f6b August 17, 2026 21:33

DAOS-17519 test: Fix the dlck_utils class.

a30908f

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 2 commits August 18, 2026 17:52

DAOS-17519 test: Address dlck_basic_fault test failure.

6031604

Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

rpadma2 added 4 commits August 20, 2026 15:46

DAOS-17519 test: Create a separate dlck logger.

9c314c7

Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

DAOS-17519 test: Removed unused variable.

04898a8

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

DAOS-17519 test: Fix the pylint issue.

903d016

Skip-func-hw-test-medium: false Skip-func-hw-test-medium-md-on-ssd: false Skip-unit-test: true Skip-fault-injection-test: true Skip-nlt: true Test-tag: DlckBasicFaultTest DlckBasicTest Signed-off-by: rpadma2 <ravindran.padmanabhan@hpe.com>

Uh oh!

Conversation

rpadma2 commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions Bot commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daosbuild3 commented Jan 8, 2026

Uh oh!

daosbuild3 commented Jan 8, 2026

Uh oh!

daosbuild3 commented Mar 12, 2026

Uh oh!

daosbuild3 commented Mar 31, 2026

Uh oh!

daosbuild3 commented May 23, 2026

Uh oh!

daosbuild3 commented May 23, 2026

Uh oh!

daosbuild3 commented Aug 12, 2026

Uh oh!

daosbuild3 commented Aug 12, 2026

Uh oh!

daosbuild3 commented Aug 13, 2026

Uh oh!

daosbuild3 commented Aug 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daltonbohning Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

rpadma2 Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

daosbuild3 commented Aug 13, 2026

Uh oh!

daosbuild3 commented Aug 13, 2026

Uh oh!

Uh oh!

shimizukko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

daosbuild3 commented Aug 17, 2026

Uh oh!

daosbuild3 commented Aug 17, 2026

Uh oh!

daosbuild3 commented Aug 18, 2026

Uh oh!

daosbuild3 commented Aug 19, 2026

Uh oh!

daosbuild3 commented Aug 19, 2026

Uh oh!

daosbuild3 commented Aug 20, 2026

Uh oh!

daosbuild3 commented Aug 20, 2026

Uh oh!

daosbuild3 commented Aug 20, 2026

Uh oh!

rpadma2 commented Dec 23, 2025 •

edited

Loading

github-actions Bot commented Dec 23, 2025 •

edited

Loading