-
Notifications
You must be signed in to change notification settings - Fork 860
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI nightlies cpu/gpu & cleanup #75
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aliberts
force-pushed
the
user/aliberts/2024_04_16_ci_builds
branch
6 times, most recently
from
April 20, 2024 16:32
bd754e2
to
af98239
Compare
aliberts
force-pushed
the
user/aliberts/2024_04_16_ci_builds
branch
2 times, most recently
from
April 24, 2024 07:08
3f2ed73
to
3b62033
Compare
aliberts
force-pushed
the
user/aliberts/2024_04_16_ci_builds
branch
from
April 25, 2024 10:21
c12712a
to
0b4277a
Compare
Cadene
approved these changes
Apr 25, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks ;)
I would suggest renaming ete
-> end-to-end
to be more explicit, but not sure.
Co-authored-by: Alexander Soare <[email protected]> Tests cleaning & simplification (#81) Fix tolerance for delta_timestamps (#84) Co-authored-by: Remi <[email protected]> Hotfix test_examples.py (#87) Quality of life patches for eval.py (#86) Add meta_data, revision v1.1 WIP add load functions + episode_data_index id -> index, finish moving compute_stats before hf_dataset push_to_hub Use v1.1, hf_transform_to_torch, Add 3 xarm datasets Remove Prod, Tests are passind Add tests/data small fix fix visualize_dataset fix online training fix online training fix online training Fixes for datasets 2.18 -> 2.19 update (#88) WIP Dockerfile Add Makefile Add cpu build Add libhdf5-dev in build-image Add gpu build Add libegl1-mesa-dev in cpu build WIP Add docker build ci Fix docker context Fix dockerfile path Fix dockerhub repo Free up more disk space on runner Include envs in cpu build remove rm -rf /usr/local/share/boost Test with pip only Add python version file Fix python version & extras Checkout pathed envs Add test artifacts Update PR template WIP Add nightly tests Push gpu & cpu images to gpu/cpu dockerhub repos WIP Test nightly WIP test cpu build WIP checkout pip install Use Makefile for end-to-end tests Fix cpu venv path Fix end-to-end Checkout envs Revert "Checkout envs" This reverts commit 7193f60. Add working-directory Fix builds (rebase hotfix #87 from main) Test fixed builds rebase from user/rcadene/2024_04_18_episode_data_index Test images Check nvidia-smi Fix opengl on gpu image Test gpu build Try nvidia/cudagl Test cudagl build Try different registry
aliberts
force-pushed
the
user/aliberts/2024_04_16_ci_builds
branch
from
April 25, 2024 12:48
feb75c4
to
9373d36
Compare
aliberts
added a commit
that referenced
this pull request
Apr 27, 2024
- Changes on the `test.yml` workflow: - Using poetry instead of pip. Contrary to what I wrote in #75, it is possible to use poetry (and have the benefits of shorter install times) without the need for having two separate versions of `pyproject.toml` and `poetry.lock`. - Reduce the trigger scope to only run when files in these directories are modified: - `lerobot/` - `tests/` - `examples/` - `.github/` - Add `style.yml` workflow for doing a `ruff check` pass on the code - More cleanup (removed deprecated workflow)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the ability to test lerobot on cpu & gpu docker builds
Changes:
pip install
rather than poetry. This is to remove the necessity to have and maintain 2 different versions ofpyproject.toml
andpoetry.lock
.This does increase install time as poetry parallelizes package installs and is much faster (~30s vs ~3mn) but dependencies will be cached so it won't matter much in most cases where the PR doesn't change the
pyproject.toml
file.pyproject.toml
files for the docker builds: I need to install the gym envs as local path dependency after checking them out with ssh in the CI, rather than as direct git dependencies during the docker build which would require setting up ssh in the dockerfile. Since I do not want to go down that rabbit hole, I've set up a script to automatically change those git dependancies as local path dependencies. When the environments will be pip-installable (at release), all this hurdle will disappear.DEVICE
flag that shows up when running tests.Makefile
to just callmake test-ete
rather than having duplicates in the different workflows.CI structure:
Todo in future PRs (at release):