Skip to content

[RFC] config: runtime: tests: ltp: enable support for code coverage #2871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

a-wai
Copy link
Contributor

@a-wai a-wai commented May 6, 2025

When enabling code coverage support using in-kernel GCOV, we must first reset coverage "recording" right before starting the test, then ensure we pack and upload the resulting artifacts.

The latter is implemented by POSTing the artifact to the storage server, providing the artifact name and URL as a LAVA test result using the following format:

artifact-upload:<artifact name>:<artifact URL>

This allows the pipeline to add the uploaded artifact(s) to the corresponding node so the information can be easily re-used at a later stage.

Depends on kernelci/kernelci-pipeline#1141

@a-wai a-wai requested review from nuclearcat, pawiecz and laura-nao May 6, 2025 16:23
@a-wai a-wai changed the title config: runtime: tests: ltp: enable support for code coverage [RFC] config: runtime: tests: ltp: enable support for code coverage May 6, 2025
@a-wai
Copy link
Contributor Author

a-wai commented May 6, 2025

Notes:

  • some of the needed information could be encoded in job configs (e.g. the upload path and name) rather than the template, but it needs some more thinking to make things more generic
  • likewise, the pre/post-processing could be template fragments to be included based on certain conditions

This PR in its current state is mostly aimed at testing how everything works together on a "real" pipeline instance, and trigger a discussion about the parts that could/should be rewritten in a more generic way.

@laura-nao
Copy link
Contributor

likewise, the pre/post-processing could be template fragments to be included based on certain conditions

Agreed - since gcov only works on kernels with the right configs and adds some overhead, it makes sense to include the pre/post processing steps conditionally.

Other than that, it looks good to me and seems ready for initial testing on staging

@a-wai
Copy link
Contributor Author

a-wai commented May 7, 2025

since gcov only works on kernels with the right configs and adds some overhead

FWIW the overhead is almost null, especially thanks to the check on L43-45:

          - GCDA=/sys/kernel/debug/gcov
          # Bail out if $GCDA doesn't exist (as GCOV is likely not enabled)
          - test -d ${GCDA} || exit 0

The idea of having included "sub"-templates would be more related to avoiding code duplication and easily enabling gcov support in different test templates, although if (and that's a big if for now) I can find a way to make the include conditional that'll be even better.

@a-wai a-wai force-pushed the add-ltp-gcov-upload branch from db7c120 to 20d3ecc Compare May 15, 2025 13:06
a-wai added 2 commits May 20, 2025 14:20
Enabling code coverage for a given test job requires 2 actions:
* reset GCOV so we only record coverage data during the test itself
* pack and upload artifacts to online storage after the test completes

The latter is implemented by POSTing the artifact to the storage server,
providing the artifact name and URL as a LAVA test result using the
following format:

  artifact-upload:<artifact name>:<artifact URL>

This allows the pipeline to add the uploaded artifact(s) to the
corresponding node so the information can be easily re-used at a later
stage.

As we'll enable coverage in many different tests, turn those actions
into separate LAVA test definitions that can then be (conditionally)
included by each test template when needed.

Signed-off-by: Arnaud Ferraris <[email protected]>
When enabling code coverage support using in-kernel GCOV, we must first
reset coverage "recording" right before starting the test, then ensure
we pack and upload the resulting artifacts.

Those steps are executed by including the corresponding sub-templates
whenever the kernel has been built with GCOV support, IOW when the
`coverage` fragment is part of the build config.

Signed-off-by: Arnaud Ferraris <[email protected]>
@a-wai a-wai force-pushed the add-ltp-gcov-upload branch from 20d3ecc to 5089ec7 Compare May 20, 2025 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants