refactor: use local git operations for commit data by Septimus4 · Pull Request #294 · tupui/soroban-versioning

Septimus4 · 2025-10-05T12:42:12Z

Migrate from Github API to a agnostic git implementation, just clone and build the necessary data

clones repositories locally, fetches commit history, and serves commit and file data without GitHub APIs
refactor state and service helpers to track repository URLs and consume the local git endpoint for history, commit data, and README retrieval
update project views to rely on the new service, handle missing external links gracefully, and adjust copy to reference generic repository history

Draft, I will update when #271 is merged

netlify · 2025-10-05T12:42:17Z

✅ Deploy Preview for staging-tansu ready!

Name	Link
🔨 Latest commit	`d925326`
🔍 Latest deploy log	https://app.netlify.com/projects/staging-tansu/deploys/68e82a7bd1ad4600084e077f
😎 Deploy Preview	https://deploy-preview-294--staging-tansu.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 75 Accessibility: 89 Best Practices: 83 SEO: 92 PWA: 80 View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2025-10-05T12:43:13Z

✅ Deploy Preview for tansu canceled.

Name	Link
🔨 Latest commit	`d925326`
🔍 Latest deploy log	https://app.netlify.com/projects/tansu/deploys/68e82a7b3aa1d100086192a5

tupui · 2025-10-06T22:51:12Z

#271 was just merged 🎉 This can be resumed.

tupui · 2025-10-09T16:08:12Z

@Septimus4 OnlyDust is closing shop. I sent you another reward now for what you did before and this.

I suggest you take out quickly until you still can.

Septimus4 · 2025-10-09T16:29:42Z

@tupui okay noted thank you

Septimus4 · 2025-10-09T21:38:56Z

@tupui here is a full v1, the main issue is that the first time we open a project the loading is on the slow side, especially for metrics, so I cache them for 5min with a in memory cache.

I can work on a smarter cache or we can have a totally different startegy.
When we are happy with the caching state I will mark it ready for review, I don't want to use ci before

tupui · 2025-10-10T03:42:59Z

A 5 mins caching sounds good to start with. I am not sure about a different strategy for now so we can start like that.

Septimus4 · 2025-10-10T06:21:51Z

should we also put a upper bound limit on how many repo can be cached at the same time ?
I am just worried many users opening many different repos will OOM the host.
It would probably require a lot of user visiting a lot of different repos.

tupui · 2025-10-10T08:10:53Z

Makes sense. Something like 10 is enough I think. But at the same time users would have to open many repos in less than 5 mins. You really need to want to do that 😅

tupui · 2025-10-10T08:12:27Z

Or do you mean by that that the solution requires a sort of backend?? Because I would want that the git clone is done on the user side. We don't want a backend as the only other place with a backend is the IPFS token and I will eventually move that to a cloudflare worker.

Septimus4 · 2025-10-10T10:01:00Z

My understanding is that you can't have a fully static site that clones a repo.
Browsers can’t run native git, and clone over HTTPS will be blocked by CORS.

I need to make more research to see if there is a simple solution but from a rapid goodle search + Chatgpt you need a more complex flow.

Cloudflare Worker proxy:

Purpose: add CORS headers so the browser can speak Git Smart HTTP to any remote.
Shape: a tiny, stateless Cloudflare Worker that forwards requests to the Git host and returns the response with Access-Control-Allow-Origin.
Set an env var ALLOWED_GIT_HOSTS (e.g., “github.com, gitlab.com, bitbucket.org”) to restrict usage

Client-only cloning:
Filesystem: LightningFS (IndexedDB/OPFS-backed) in the browser.
HTTP: isomorphic-git (lib) with corsProxy pointing to your Worker.
(Most infos from https://isomorphic-git.org/)

The main limitation since the browser is doing the cloning and you need valid CORS. Having a cors proxy kind of defeats the initial idea, you only support whitelisted domains so not very agnostic, and you have complex setup of Filesystem in the browser.
In that case it looks lighter and easier to just support the main git providers API, Github, Gitlab, Bitbucket and Codeberg.

But let me make some in depth research to make sure there is no light solution possible

tupui · 2025-10-10T10:09:42Z

Thanks 🙏 otherwise we could start with some allow list or have a proxy on cloudflare for that.

Septimus4 · 2025-10-13T19:45:51Z

From my research, it should be possible to fetch publicly available repositories from any provider that supports the Git Smart HTTP protocol and exposes endpoints with permissive or flexible CORS policies.
This would allow the app to clone, inspect, or retrieve repository data directly in the browser without authentication, backend servers, or provider-specific APIs.

Providers to Support in v1

Provider	Protocol	CORS Behavior	Expected Support
GitHub	Smart HTTP, REST API	CORS-friendly on API and codeload domains	Full support
GitLab.com	Smart HTTP, REST API	Public projects allow unauthenticated HTTP access; API has permissive CORS	Full support
Codeberg / Gitea / Forgejo	Smart HTTP, REST API	Depends on instance config; most public instances allow basic CORS	Partial support

Implementation Plan (v1)

Repository URL Normalization
- Parse and standardize user-provided URLs into {host, owner, repo, ref}.
- Infer default branch if not provided.
Provider Adapter Layer
- Implement an interface per provider exposing:
  - getHead(ref) → returns { sha, author, date, message }
  - getArchive(ref) → returns ArrayBuffer (zip or tarball)
- Dispatch adapters by hostname pattern.
Metrics Computation
- Decompress archives in-browser (fflate or JSZip).
- Compute file/LOC/size metrics locally.
- Optionally derive a reproducible content hash (SHA-256 over sorted file entries).
Caching and Rate Limits
- Cache results in IndexedDB keyed by {host, repo, ref, sha}.
- Respect rate limits.

tupui · 2025-10-14T06:09:36Z

Great to read!

tupui · 2025-10-23T12:37:59Z

Hey 👋 just checking, did you get any time to make some progress?

Septimus4 · 2025-10-23T12:40:51Z

Yes I have a branch locally with the implementation, I need to test it more fix some edge cases, but I will open the PR this weekend

tupui · 2025-10-23T12:55:10Z

Awesome, thanks for the update 🚀

- Add ALLOWED_GIT_HOSTS allow list to local git API (github, gitlab, bitbucket, codeberg, etc.) - Support additional hosts via GIT_ALLOWED_HOSTS environment variable - Create git-proxy Cloudflare Worker as alternative to local API - Add PUBLIC_GIT_PROXY_URL environment variable to switch between local API and Cloudflare proxy - Worker supports GitHub, GitLab, Bitbucket, Codeberg APIs with unified interface - Add CORS and security controls to both local and proxy implementations

netlify · 2026-01-13T15:55:13Z

👷 Deploy request for tansu pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	`a0e1879`

netlify · 2026-01-13T15:55:15Z

❌ Deploy Preview for staging-tansu failed. Why did it fail? →

Name	Link
🔨 Latest commit	`a0e1879`
🔍 Latest deploy log	https://app.netlify.com/projects/staging-tansu/deploys/69666ae08859960008ea68de

tupui · 2026-01-17T17:15:18Z

Nice to see some movement :)

tupui · 2026-01-19T10:39:26Z

Do you have some updates @Septimus4? This could be part of the new Wave program.

Septimus4 · 2026-01-19T11:12:05Z

I am close to initial testing, if it goes well it could be ready soon. It should, but I will only be certain when I run it end to end. It's more change that I initally imagined.

tupui · 2026-01-19T11:23:24Z

Awesome 🙌

tupui · 2026-03-02T12:04:35Z

Hey 👋 any updates?

Septimus4 marked this pull request as draft October 5, 2025 12:43

tupui linked an issue Oct 5, 2025 that may be closed by this pull request

History without GitHub #81

Open

Septimus4 force-pushed the agnostic-git branch from 37c785d to 7e0e659 Compare October 9, 2025 19:35

Septimus4 mentioned this pull request Oct 27, 2025

Agnostic git v2 #329

Closed

Septimus4 added 3 commits January 13, 2026 14:10

feat: agnostic git support

22947ba

perf: make initial load faster and cache repo 5min

c4ac9cb

Septimus4 force-pushed the agnostic-git branch from d925326 to a0e1879 Compare January 13, 2026 15:55

Conversation

Septimus4 commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify Bot commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for staging-tansu ready!

Uh oh!

netlify Bot commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for tansu canceled.

Uh oh!

tupui commented Oct 6, 2025

Uh oh!

tupui commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Septimus4 commented Oct 9, 2025

Uh oh!

Septimus4 commented Oct 9, 2025

Uh oh!

tupui commented Oct 10, 2025

Uh oh!

Septimus4 commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tupui commented Oct 10, 2025

Uh oh!

tupui commented Oct 10, 2025

Uh oh!

Septimus4 commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tupui commented Oct 10, 2025

Uh oh!

Septimus4 commented Oct 13, 2025

Providers to Support in v1

Implementation Plan (v1)

Uh oh!

tupui commented Oct 14, 2025

Uh oh!

tupui commented Oct 23, 2025

Uh oh!

Septimus4 commented Oct 23, 2025

Uh oh!

tupui commented Oct 23, 2025

Uh oh!

netlify Bot commented Jan 13, 2026

👷 Deploy request for tansu pending review.

Uh oh!

netlify Bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ Deploy Preview for staging-tansu failed. Why did it fail? →

Uh oh!

tupui commented Jan 17, 2026

Uh oh!

tupui commented Jan 19, 2026

Uh oh!

Septimus4 commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tupui commented Jan 19, 2026

Uh oh!

tupui commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Septimus4 commented Oct 5, 2025 •

edited

Loading

netlify Bot commented Oct 5, 2025 •

edited

Loading

netlify Bot commented Oct 5, 2025 •

edited

Loading

tupui commented Oct 9, 2025 •

edited

Loading

Septimus4 commented Oct 10, 2025 •

edited

Loading

Septimus4 commented Oct 10, 2025 •

edited

Loading

netlify Bot commented Jan 13, 2026 •

edited

Loading

Septimus4 commented Jan 19, 2026 •

edited

Loading