-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Git LFS instructions #133
Open
melody-ren
wants to merge
14
commits into
NVIDIA:main
Choose a base branch
from
melody-ren:lfs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
514797c
Add test data for LFS
melody-ren a63730d
Add another mock data for lfs
melody-ren a088fab
remove a lfs file
melody-ren 8a5cbbb
Add instructions on how to set up Git LFS
melody-ren b6158b7
Format
melody-ren b7257d2
Merge branch 'main' into lfs
melody-ren f40f533
Add file size and type check to workflow
melody-ren 71db73d
Merge branch 'main' into lfs
melody-ren 4d21881
Fetch base branch for comparison
melody-ren b7291e3
Merge branch 'lfs' of github.com:melody-ren/cudaqx into lfs
melody-ren bca99db
DCO Remediation Commit for Melody Ren <[email protected]>
melody-ren 3f93e32
[TEST CI] expect pipeline to fail
melody-ren bfbc30b
[TEST CI] expect pipeline to pass
melody-ren e418aba
Remove untracked file from .gitattributes
melody-ren File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
assets/datasets/* filter=lfs diff=lfs merge=lfs -text |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
name: Check for Large Files and Restricted Extensions | ||
|
||
on: | ||
pull_request: | ||
branches: | ||
- main | ||
types: [opened, synchronize, reopened] | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
env: | ||
LLVM_VERSION: 16 | ||
|
||
jobs: | ||
check-files: | ||
name: Check file size and type | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
set-safe-directory: true | ||
fetch-depth: 1 | ||
|
||
- name: Fetch base branch | ||
run: git fetch origin ${{ github.event.pull_request.base.ref }} --depth=1 | ||
|
||
- name: Check for large files | ||
run: | | ||
MAX_SIZE=5M # Set max file size limit | ||
LARGE_FILES=$(git diff --name-only --diff-filter=A origin/${{ github.event.pull_request.base.ref }} | xargs du -h | awk -v max="$MAX_SIZE" '$1 > max {print $2}') | ||
|
||
if [[ ! -z "$LARGE_FILES" ]]; then | ||
echo "❌ The following files exceed the allowed size of $MAX_SIZE:" | ||
echo "$LARGE_FILES" | ||
exit 1 | ||
fi | ||
|
||
- name: Check for restricted file types | ||
run: | | ||
BLOCKED_EXTENSIONS="(exe|zip|tar.gz|bz2)" # Add any forbidden extensions | ||
BAD_FILES=$(git diff --name-only --diff-filter=A origin/${{ github.event.pull_request.base.ref }} | grep -E "\.($BLOCKED_EXTENSIONS)$" || true) | ||
if [[ ! -z "$BAD_FILES" ]]; then | ||
echo "❌ The following files have restricted extensions:" | ||
echo "$BAD_FILES" | ||
exit 1 | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Git LFS Setup and Usage Guide | ||
|
||
## Installation | ||
|
||
Git LFS must be installed before using it in a repository. Follow the installation steps based on your operating system. | ||
|
||
### Ubuntu (Debian-based distributions) | ||
```sh | ||
sudo apt update | ||
sudo apt install git-lfs | ||
git lfs install | ||
|
||
### AlmaLinux and ManyLinux | ||
sudo dnf install git-lfs | ||
git lfs install | ||
|
||
### macOS | ||
brew install git-lfs | ||
git lfs install | ||
``` | ||
|
||
## Tracking and committing large files | ||
|
||
1. Initialize Git LFS in your repository: | ||
|
||
`git lfs install` | ||
|
||
2. Track specific file types or individual files using the following command: | ||
|
||
`git lfs track "assets/*"`, where `assets` is a directory containing large files. | ||
|
||
3. Commit the changes to `.gitattributes`: | ||
|
||
`git add .gitattributes && git commit -m "Track large files with Git LFS"` | ||
|
||
4. Add and commit the large files: | ||
|
||
`git add assets/largefile.zip && git commit -m "Add large file"` | ||
|
||
5. Push to remote: | ||
|
||
`git push origin branch_name` | ||
|
||
## Cloning and fetching large files | ||
|
||
1. Clone a repository that uses Git LFS: | ||
|
||
`git clone https://github.com/username/repository.git`. By default, cloning only retrieves the pointer files to the large file. To fetch the actual large files, use `git lfs pull`. | ||
|
||
2. Fetch large files for an existing repository: | ||
|
||
`git lfs pull` | ||
|
||
## Check Git LFS status | ||
|
||
To check which files are tracked by Git LFS: | ||
|
||
`git lfs ls-files` | ||
|
||
## Removing a file from LFS | ||
|
||
Use the following steps to remove a file from LFS: | ||
|
||
`git rm --cached assets/largefile.zip`, then commit and push. | ||
|
||
Once the file is removed, remember to delete the tracking information in `.gitattributes`. | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I set this to be 5M for now. Happy to change to some other more sensible numbers