Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control aloha robot natively #316

Merged
merged 45 commits into from
Sep 4, 2024
Merged

Conversation

Cadene
Copy link
Collaborator

@Cadene Cadene commented Jul 11, 2024

What this does

  • Refactor koch.py by replacing it with more general manipulator.py
  • Add lerobot/configs/robot/aloha.yaml
  • Add robot_type (e.g. koch, aloha) which enable custom robot settings (e.g. torque in leader trigger for koch)
  • Fix unit tests for robot devices
  • Add safety layer for Aloha by default
  • Improve calibration code
    • Add CalibrationMode enum:
      • joints in DEGREE mode come with a nominal range [-180, +180] degrees
      • joints in LINEAR mode come with a nominal range [0, 100] % with 0% fully close, 100% fully open (used by Aloha gripper)
    • Replace pickle calibration file, by 1 json calibration file per arm (allows to recalibrate only one arm if needed) ---> BREAKING CHANGE: everyone will need to recalibrate <---
    • Auto adjust calibration when a jump of 4096 motor steps is detected (fix a common bug requiring recalibration with Koch robot)
    • Add images for manual calibration of Aloha (note: for Aloha, manual calibration is not needed ; it is triggered only if a motor was replaced and wrongly positioned)
zero rotated rest
follower_zero follower_rotated follower_rest
leader_zero leader_rotated leader_rest

TODO next PR:

How it was tested

On Koch + Koch bimanual + Aloha:

  • Ran calibration
  • Ran teleoperation
  • Ran data recording (with 4 cameras for Aloha)
  • Ran data visualization
  • Ran replay

Also re-read the tutorial to make sure it is up-to-date and compatible with Aloha

How to checkout & try? (for the reviewer)

Koch

python lerobot/scripts/control_robot.py record \
    --robot-path lerobot/configs/robot/koch.yaml \
    --fps 30 \
    --root /tmp/data \
    --repo-id $USER/test \
    --num-episodes 1 \
    --run-compute-stats 0 \
    --warmup-time-s 2 \
    --episode-time-s 30 \
    --reset-time-s 1 \
    --force-override 1 \
    --push-to-hub 0

python lerobot/scripts/visualize_dataset_html.py \
  --root /tmp/data \
  --repo-id $USER/test

python lerobot/scripts/control_robot.py replay \
    --robot-path lerobot/configs/robot/koch.yaml \
    --fps 30 \
    --root /tmp/data \
    --repo-id $USER/test

Aloha

python lerobot/scripts/control_robot.py record \
    --robot-path lerobot/configs/robot/aloha.yaml \
    --fps 30 \
    --root /tmp/data \
    --repo-id $USER/test \
    --num-episodes 1 \
    --run-compute-stats 0 \
    --warmup-time-s 2 \
    --episode-time-s 30 \
    --reset-time-s 1 \
    --force-override 1 \
    --push-to-hub 0

python lerobot/scripts/visualize_dataset_html.py \
  --root /tmp/data \
  --repo-id $USER/test

python lerobot/scripts/control_robot.py replay \
    --robot-path lerobot/configs/robot/aloha.yaml \
    --fps 30 \
    --root /tmp/data \
    --repo-id $USER/test

@Cadene Cadene changed the base branch from main to user/rcadene/2024_06_22_control_robot July 11, 2024 13:22
@Cadene Cadene force-pushed the user/rcadene/2024_07_11_control_aloha branch from 9b7fb05 to 4044fa9 Compare July 15, 2024 12:56
Base automatically changed from user/rcadene/2024_06_22_control_robot to main July 15, 2024 15:43
@Cadene Cadene force-pushed the user/rcadene/2024_07_11_control_aloha branch from 4044fa9 to cd927e1 Compare August 1, 2024 14:42
@Cadene Cadene changed the base branch from main to user/rcadene/2024_07_16_control_robot_v2 August 1, 2024 14:42
Base automatically changed from user/rcadene/2024_07_16_control_robot_v2 to main August 15, 2024 16:11
@Cadene Cadene closed this Aug 22, 2024
@Cadene Cadene force-pushed the user/rcadene/2024_07_11_control_aloha branch from 2d9c06b to b5ad79a Compare August 22, 2024 15:49
@Cadene Cadene reopened this Aug 22, 2024
@Cadene Cadene marked this pull request as ready for review August 22, 2024 16:09
@Cadene Cadene requested a review from alexander-soare August 23, 2024 12:36
Copy link
Contributor

@alexander-soare alexander-soare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I'm going a bit on trust that you've tested this thoroughly as I don't have a robot to try it with.

If there's something you'd like me too look at more deeply, please draw my attention to it.

@Cadene Cadene requested a review from aliberts August 29, 2024 22:45
Copy link
Contributor

@alexander-soare alexander-soare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved based on the parts that I was tagged in. Allowing @aliberts to lead this review.

Copy link
Collaborator

@aliberts aliberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great additions, thanks @Cadene.

LGTM with 2 caveats:

  • I couldn't try and run the code on any robots as I don't have access rn
  • My review is rather shallow as I understand you that want this to be merged quickly

Left some comments, mostly non-blocking but with some questions

lerobot/configs/robot/aloha.yaml Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/utils.py Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
Copy link
Collaborator Author

@Cadene Cadene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/utils.py Show resolved Hide resolved
lerobot/configs/robot/aloha.yaml Outdated Show resolved Hide resolved
lerobot/common/robot_devices/robots/manipulator.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@aliberts aliberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again!

lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/motors/dynamixel.py Outdated Show resolved Hide resolved
lerobot/common/robot_devices/utils.py Outdated Show resolved Hide resolved
@Cadene Cadene merged commit 429a463 into main Sep 4, 2024
10 checks passed
@Cadene Cadene deleted the user/rcadene/2024_07_11_control_aloha branch September 4, 2024 17:28
amandip7 pushed a commit to amandip7/lerobot that referenced this pull request Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants