Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor OpenX #505

Merged

Conversation

michel-aractingi
Copy link
Collaborator

What this does

Simplifies OpenX files by focusing only on the RLDS format. Remove unnecessary file from OpenVLA for training with OpenX. Now all RLDS datasets (OpenX and others) can use this format freely.

Adds missing features and observations from the RLDS format to LeRobotDataset, e.g., is_first, is_terminal, discount...

Example:

 python lerobot/scripts/push_dataset_to_hub.py \
        --raw-dir /fsx/michel_aractingi/datasets/maniskill_dataset_converted_externally_to_rlds/0.1.0 \
        --repo-id lerobot/maniskill_dataset \
        --raw-format rlds 

How it was tested

Visualize datasets to make sure they were pushed correctly

python lerobot/scripts/visualize_dataset_html.py --repo-id lerobot/maniskill_dataset --episodes 0 1 2 3 4

NOTE: will only push after LeRobotDatasetV2 is merged.

@michel-aractingi michel-aractingi force-pushed the user/michel-aractingi/13-11-2024-refactor-openx branch from 7f3444c to ad2cdaf Compare December 1, 2024 21:34
@michel-aractingi michel-aractingi marked this pull request as ready for review December 2, 2024 13:15
Copy link
Collaborator

@Cadene Cadene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to merge. However, this code is obsolete since Dataset V2 merged.

When we have time, we should port this code to examples/port_datasets and delete push_dataset_to_hub folder + script.

@michel-aractingi michel-aractingi merged commit a2c1819 into main Dec 2, 2024
7 checks passed
@michel-aractingi michel-aractingi deleted the user/michel-aractingi/13-11-2024-refactor-openx branch December 2, 2024 23:51
helper2424 pushed a commit to helper2424/lerobot that referenced this pull request Dec 17, 2024
villekuosmanen added a commit to villekuosmanen/lerobot that referenced this pull request Dec 30, 2024
* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484)

Co-authored-by: Alexander Soare <[email protected]>

* Fix config file (huggingface#495)

* fix: broken images and a few minor typos in README (huggingface#499)

Signed-off-by: ivelin <[email protected]>

* Add support for Windows (huggingface#494)

* bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450)

* Add distinction between two unallowed cases in name check "eval_" (huggingface#489)

* Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490)

* Dataset v2.0 (huggingface#461)

Co-authored-by: Remi <[email protected]>

* Refactor OpenX (huggingface#505)

* Fix missing local_files_only in record/replay (huggingface#540)

Co-authored-by: Simon Alibert <[email protected]>

* Control simulated robot with real leader (huggingface#514)

Co-authored-by: Remi <[email protected]>

* Update 7_get_started_with_real_robot.md (huggingface#559)

* LerobotDataset pushable to HF from any folder (huggingface#563)

* Fix example 6 (huggingface#572)

* fixing typo from 'teloperation' to 'teleoperation' (huggingface#566)

* [vizualizer] for LeRobodDataset V2 (huggingface#576)

* Fix broken `create_lerobot_dataset_card`  (huggingface#590)

* feat(act): support training end of episode token to ACT model

* changes

* feat(arx): add arx arm (#2)

* feat(arx): support arx arm

* changes

* changes

* changes

* changes

* pass pipes explicitly

* changes

* us ndarray over a pipe

* changes

* changes

* replay basically works

* patch arx sdk

* changes

* support cameras in arx5

* rename to arx5

* kind of works

* changes

* changes

* changes

* various changes

* changes

* revert a few changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* remove TODO

* allow multiple tasks

---------

Signed-off-by: ivelin <[email protected]>
Co-authored-by: Hirokazu Ishida <[email protected]>
Co-authored-by: Alexander Soare <[email protected]>
Co-authored-by: Arsen Ohanyan <[email protected]>
Co-authored-by: Ivelin Ivanov <[email protected]>
Co-authored-by: Daniel Ritchie <[email protected]>
Co-authored-by: resolver101757 <[email protected]>
Co-authored-by: Jannik Grothusen <[email protected]>
Co-authored-by: KasparSLT <[email protected]>
Co-authored-by: Simon Alibert <[email protected]>
Co-authored-by: Remi <[email protected]>
Co-authored-by: Michel Aractingi <[email protected]>
Co-authored-by: Simon Alibert <[email protected]>
Co-authored-by: berjaoui <[email protected]>
Co-authored-by: Claudio Coppola <[email protected]>
Co-authored-by: s1lent4gnt <[email protected]>
Co-authored-by: Mishig <[email protected]>
Co-authored-by: Eugene Mironov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants