Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase download limits for caching to enable fine grained caching #738

Open
newhoggy opened this issue Mar 16, 2021 · 0 comments
Open
Labels
cache enhancement New feature or request

Comments

@newhoggy
Copy link

newhoggy commented Mar 16, 2021

First some background

I have implemented fine grained caching for Haskell projects where project dependencies are cached individually and it works really well for small projects.

The action is here:

https://github.com/action-works/cabal-cache

For an example of a build using this see:

https://github.com/haskell-works/hw-prim/actions/runs/651341909

The problem arises when I try to use it on a large project with hundreds of dependencies:

https://github.com/input-output-hk/cardano-node/runs/2112862696?check_suite_focus=true

In build(8.10.3, windows-latest) under the step Cache cabal store, things look like they are going find until at some point I start getting lots of these errors:

Warning: Cache service responded with 429

Looks like there is some kind of throttling.

Describe the enhancement

Please provide fine grained caching of files without throttling, or at least increase the limit to the throttling by maybe an order of magnitude.

Note, I have tested this against S3, and this is very fast, but unfortunately use of S3 in a secure and convenient way is not fork friendly.

Code Snippet
N/A

Additional information
N/A

@newhoggy newhoggy added the enhancement New feature or request label Mar 16, 2021
@thboop thboop added the cache label Apr 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cache enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants