HelpersTask487_Add_Git_actions_for_Codecov_3 #516

Shaunak01 · 2025-04-08T23:15:23Z

codecov · 2025-04-09T21:21:18Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Shaunak01 · 2025-04-09T21:27:58Z

@gpsaggese @dremdem

Take a look at how the coverage looks: https://app.codecov.io/github/causify-ai/helpers/tree/HelpersTask487_Add_Git_actions_for_Codecov_3/

We get a 14 day free trial for complete org so we can do that if we like this service.

.github/gh_requirements.txt

.github/workflows/test_coverage.yml

.github/gh_requirements.txt

gpsaggese · 2025-04-09T22:51:40Z

helpers/test/test_lib_tasks_gh.py

-        not hgit.is_in_helpers_as_supermodule(),
-        reason="""Skip unless helpers is the supermodule. Fails when updating submodules;
-            passes in fast tests super-repo run. See CmTask10845.""",
+        not hgit.is_in_helpers_as_supermodule() or os.getenv("CSFY_CI") == "true",


there is a is_ci function somewhere in hserver

samarth9008 · 2025-04-09T23:05:13Z

@dremdem

Let me know if you have any blockages.

@Shaunak01

The working should be similar to the current coverage run

dremdem

For draft PR is looking good.
Small nits.
Considering
https://github.com/causify-ai/helpers/pull/516/files#r2036238705
Need to rework it as separate GH actions workflows for fast, slow and superslow.

.github/workflows/test_coverage.yml

helpers/lib_tasks_pytest.py

helpers/test/test_lib_tasks_gh.py

Shaunak01 · 2025-04-12T00:28:05Z

@dremdem

I am trying to generate separate and combined coverage reports for fast and slow tests to see sep reports on copdecov. Initially, the default approach worked by using the implicit normalization of file paths in the default .coverage file, but in my approach we renamed the files and explicitly set the data file with the COVERAGE_FILE variable, absolute paths (like /app/config_root/__init__.py and /app/conftest.py) began causing errors because Coverage couldn't find the corresponding local source. Now, I am exploring ways to either map these paths, ignore errors, or otherwise replicate the default behavior so that every file is included in the report without manual omission.

….com:causify-ai/helpers into HelpersTask487_Add_Git_actions_for_Codecov_3

Shaunak01 · 2025-04-14T23:54:21Z

@gpsaggese @dremdem

Pls take a look at: https://app.codecov.io/github/causify-ai/helpers

We can have the coverage for just the fast tests, slow tests or you can select both flags to see combined

FYI the combined and helpers flag are useless so ignore

dremdem · 2025-04-15T14:03:20Z

@gpsaggese @dremdem

Pls take a look at: https://app.codecov.io/github/causify-ai/helpers

We can have the coverage for just the fast tests, slow tests or you can select both flags to see combined

FYI the combined and helpers flag are useless so ignore

It looks good.
Probably later we can also integrate it into PRs statuses:
https://github.com/marketplace/codecov

dremdem · 2025-04-15T14:22:37Z

@Shaunak01
Is there the way to fast disabling there warnings:

It's really hard to read the code between these warnings.

dremdem

Looking good.
Some questions.

dremdem · 2025-04-15T14:06:13Z

.github/workflows/test_coverage.yml

+      # After the test run, rename the generated coverage.xml to coverage_fast.xml.
+      - name: Rename Fast Test Coverage report
+        if: always()
+        run: mv ./coverage.xml ./coverage_fast.xml


Wondering, why we need it if we will specify the flags: fast in the next step?

So if you look at the invoke function in helpers/lib_tasks_pytest.py. Initially we had combined coverage only now i have introduced separate fast and slow coverage runs. But both generate the same file coverage.xml due to internal logic of porting. If i rename in the internal logic, we have to debug some porting logic as codecov is not picking up other files. So a better solution is

Generate Fast test coverage - coverage.xml -> rename 'coverage_fast.xml'-> upload
Generate Slow test coverage - coverage.xml -> rename 'coverage_slow.xml'-> upload

Codecov combines so we dont need to.

I'm not sure I got the point. To be honest, I'm lost.
@Shaunak01 Let's chat tomorrow.

dremdem · 2025-04-15T14:07:39Z

.github/workflows/test_coverage.yml

+          fail_ci_if_error: true
+
+      # This step is used to trigger the slow test coverage generation using the invoke task.
+      - name: Run Slow test and generate report


Probably, we need separate workflows for the fast, slow and super slow tests.

The previous comment answers this

gpsaggese · 2025-04-22T00:52:35Z

helpers/lib_tasks_pytest.py

+    :param target_dir: directory for coverage stats. Use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where:
+             - include_in_report is a glob pattern for files to include
+             - exclude_from_report is a comma-separated glob pattern for files to exclude (or None if not applicable)


gpsaggese · 2025-04-22T00:53:49Z

helpers/lib_tasks_pytest.py

@@ -864,6 +865,8 @@ def run_coverage_report(  # type: ignore
        if exclude_from_report is not None:
            report_html_cmd += f" --omit={exclude_from_report}"
        report_cmd.append(report_html_cmd)
+    # Generate an XML report for Codecov.
+    report_cmd.append("coverage xml -o coverage.xml")


Can we disable these comments

from Codecov?

I think they are cool, but for now they add a lot of noise.

Shaunak01 · 2025-04-22T14:09:43Z

closing with follow up PR at: #598

Shaunak01 self-assigned this Apr 8, 2025

Shaunak01 force-pushed the HelpersTask487_Add_Git_actions_for_Codecov_3 branch from 174b5f5 to 90a7090 Compare April 9, 2025 03:47

Shaunak01 mentioned this pull request Apr 9, 2025

Add Git actions for Codecov #487

Open

3 tasks

Shaunak01 force-pushed the HelpersTask487_Add_Git_actions_for_Codecov_3 branch from 15de243 to 6693bd5 Compare April 9, 2025 21:00

Shaunak01 requested review from gpsaggese and dremdem April 9, 2025 21:28

gpsaggese reviewed Apr 9, 2025

View reviewed changes

dremdem reviewed Apr 10, 2025

View reviewed changes

Shaunak01 force-pushed the HelpersTask487_Add_Git_actions_for_Codecov_3 branch from f10095d to 8a51bda Compare April 12, 2025 00:24

Shaunak01 added 4 commits April 12, 2025 02:50

Add GitHub Actions and Codecov setup

597c5da

checkpoint

298c45a

checkpoint

a622af3

checkpoint

7e60ad5

Shaunak01 force-pushed the HelpersTask487_Add_Git_actions_for_Codecov_3 branch from 924b3d6 to 7e60ad5 Compare April 12, 2025 02:53

Shaunak01 and others added 12 commits April 12, 2025 02:59

checkpoint

2452b8f

test

e591957

test

3a34b95

test

4a06935

add slow tests

8e0c0bf

checkpoint

5210cf6

resolve conflicts

bd3ed10

Merge branch 'master' into HelpersTask487_Add_Git_actions_for_Codecov_3

4e464a8

Merge branch 'master' into HelpersTask487_Add_Git_actions_for_Codecov_3

bdc2f1b

skip tests

9936d80

Merge branch 'HelpersTask487_Add_Git_actions_for_Codecov_3' of github…

054d04c

….com:causify-ai/helpers into HelpersTask487_Add_Git_actions_for_Codecov_3

checkpoint

cb12e76

Shaunak01 added 2 commits April 15, 2025 00:28

resolve comments

99a99c1

checkpoint

a4116ca

dremdem reviewed Apr 15, 2025

View reviewed changes

gpsaggese reviewed Apr 22, 2025

View reviewed changes

Shaunak01 closed this Apr 22, 2025

Shaunak01 deleted the HelpersTask487_Add_Git_actions_for_Codecov_3 branch April 22, 2025 14:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HelpersTask487_Add_Git_actions_for_Codecov_3 #516

HelpersTask487_Add_Git_actions_for_Codecov_3 #516

Shaunak01 commented Apr 8, 2025 •

edited

Loading

codecov bot commented Apr 9, 2025

Shaunak01 commented Apr 9, 2025

gpsaggese Apr 9, 2025

samarth9008 commented Apr 9, 2025

dremdem left a comment

Shaunak01 commented Apr 12, 2025

Shaunak01 commented Apr 14, 2025 •

edited

Loading

dremdem commented Apr 15, 2025

dremdem commented Apr 15, 2025

dremdem left a comment

dremdem Apr 15, 2025

Shaunak01 Apr 15, 2025

dremdem Apr 15, 2025

Shaunak01 Apr 15, 2025

dremdem Apr 15, 2025

Shaunak01 Apr 15, 2025

gpsaggese Apr 22, 2025

gpsaggese Apr 22, 2025

Shaunak01 commented Apr 22, 2025

HelpersTask487_Add_Git_actions_for_Codecov_3 #516

HelpersTask487_Add_Git_actions_for_Codecov_3 #516

Conversation

Shaunak01 commented Apr 8, 2025 • edited Loading

codecov bot commented Apr 9, 2025

Welcome to Codecov 🎉

Shaunak01 commented Apr 9, 2025

Choose a reason for hiding this comment

samarth9008 commented Apr 9, 2025

dremdem left a comment

Choose a reason for hiding this comment

Shaunak01 commented Apr 12, 2025

Shaunak01 commented Apr 14, 2025 • edited Loading

dremdem commented Apr 15, 2025

dremdem commented Apr 15, 2025

dremdem left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shaunak01 commented Apr 22, 2025

Shaunak01 commented Apr 8, 2025 •

edited

Loading

Shaunak01 commented Apr 14, 2025 •

edited

Loading