Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use raw buffer to parse files #97

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Use raw buffer to parse files #97

wants to merge 3 commits into from

Conversation

PaperStrike
Copy link

The main cause of #94 is unpkg (or the whole cloudflare, not sure) doesn't reply Content-Length anymore. Considering Content-Length isn't a reliable header nowadays, this PR enables badge-size to actually download almost all required files, which may significantly increase the bandwidth usage. Thus I implemented #16 at the same time.

The download threshold temporarily being set to 5 megabytes.

We have to find the best threshold before this is merged. 🤔

Demo: https://badge-size-teal.vercel.app/

@PaperStrike
Use Buffer.byteLength to parse file size
913eb70 
- Fix the version number of target URL in 'broken absolute' test case
- Support download size limit
- Temporary set the download limit to 5 MB
@PaperStrike
Support raw buffer
284e838 
- Update got to 11.8.2
@PaperStrike
Clean up comments on DOWNLOAD_LIMIT
e4ada78
@ngryman
Copy link
Owner

ngryman commented Sep 10, 2021

Hey @PaperStrike, thanks for the PR 🙌

I'm currently moving house, hence the delay in my answer, sorry about that.

I would need to think about the implications of dropping the HEAD HTTP method. Currently, the service already costs me quite a bit of 💰 and I'm concerned that downloading every file's payload would drastically increase my bills 😬

I'll probably dive into this during the course of this month. I might also consider an alternative of Vercel for hosting this project to reduce the costs.

@PaperStrike
Copy link
Author

It is worth to mention that if the raw sizes of files in npm packages matter significantly (I have no idea the amount of these users), we can try to read them from unpkg's official ?meta, or/and jsDelivr's official data.jsdelivr.com service. It's more reliable and faster.

@ngryman
Copy link
Owner

ngryman commented Oct 16, 2021

👋 Sorry for the lack of news.

Just a heads up that I've started a Rust implementation in the #rust branch.
It should improve a bunch of things. It's not relying on shields.io anymore so it should be more performant as well 🚀

I would like to switch to it during the next month of so 🤞

@ngryman ngryman mentioned this pull request Nov 10, 2021
Draft
@ngryman
Copy link
Owner

ngryman commented Nov 10, 2021

Alright, I have the Rust version running.
Could you take a look at #100? If everything goes well, I expect to push it to production by EOW. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Absolute url no longer supported? Limit file size
2 participants
@PaperStrike @ngryman