Skip to content

Configure Codecov for coverage #597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Shaunak01 opened this issue Apr 21, 2025 · 4 comments
Open

Configure Codecov for coverage #597

Shaunak01 opened this issue Apr 21, 2025 · 4 comments
Assignees

Comments

@Shaunak01
Copy link
Contributor

Follow up from : #487

Specs:

  1. Run the coverage once every week for the 3 builds (fast, slow and superslow).
  2. Is there a way to merge the 3 coverages to get the total one? (Look into slection of the flags to combine reports on codecov directly)
  3. Let's replace the old coverage system, adding a link in the BM view (@dremdem pls guide to this once we are done with other specs)
  4. Let's put together a short doc of how the system works, etc.
  5. Let's start with helpers and then we extend to the other repos.
  6. We can report a break if the coverage decreases more than 1% in a week. Ideally we want to only have coverage increase. (IMO lets leave this for last)

FYI @dremdem @gpsaggese

@Shaunak01
Copy link
Contributor Author

PR: #598

  • Fast, slow and combined test coverage: Everday
  • Superslow test : Everyweek
  • Fails if combined tests coverage drop by 1%
  • link: Link to dashboard
  • Once merged master coverage should be visible on codecov\

@dremdem
Copy link
Contributor

dremdem commented Apr 22, 2025

3. Let's replace the old coverage system, adding a link in the BM view (@dremdem pls guide to this once we are done with other specs)

Just need to add the link into the
https://github.com/causify-ai/helpers/blob/3c03e71d8dde66f24ba7b6f46d48274a7c671a6d/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb

Section: Code coverage HTML-page

@Shaunak01
Copy link
Contributor Author

Shaunak01 commented Apr 24, 2025

  • Now we have 3 flags - fast, slow, superslow

All can be viewed at: https://app.codecov.io/gh/causify-ai/helpers/tree/HelpersTask597_Configure_Codecov_for_coverage

  • @gpsaggese to remove combined_fast_slow
    • Reason: Codecov gives us the total coverage percentage and an option to combine different flags on the UI itself.

As discussed,
- Fast and slow test coverage will run Everyday
- Superslow will run once a week
- Comments are turned off
- Everyday a check will happen if the coverage drops by 1% (including all three tests)

@gpsaggese
Copy link
Contributor

What's left to do? See list above?

  • Document?
  • Add to the BM routine?
  • Then release to all the other repos?
  • What else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants