Skip to content

[LIBERO] pi0_pick can return success for an empty grasp #243

Description

@FM-Agent-Organization

Summary

pi0_pick returns success=true after the end effector descends and rises while the gripper opening is below a threshold. Those conditions can hold when the fingers close on air. The result therefore does not establish that an object has been picked up, and can mislead subsequent transport and release decisions.

The current docstring does describe this heuristic, and the evaluation prompt explicitly says that success is a hint rather than proof. This report concerns a reproducible false positive in the pickup heuristic and its interface design, not a violation of a current contact-based grasp guarantee. It may reasonably be triaged as a behavior improvement rather than a regression.

Version and source

Checked revision: 068cd64f4f175179a13c67d429a4d00ca56fb196, 2026-10-07. Implementation.

The current code still uses descent_done and ascended and closed, with default gripper_open_thresh=0.0, to set success. It has neither a contact query nor a held_objects result. Experimental contact-based fixes have not been incorporated into this source.

Minimal reproduction of the result logic

The companion probe_current.py loads the actual pi0_pick function and replaces only its VLA/environment boundary. With default thresholds, supply:

Observation EEF z in metres Gripper opening in metres Episode done
Initial 1.00 0.080 false
After chunk 1 0.85 0.002 false
After chunk 2 0.91 0.002 false

The function returns success=true with no object held by the controlled environment. This probe demonstrates the decision rule; physical evidence comes from the recorded run below.

Recorded physical example

LIBERO-PRO, libero_object, task 4, seed 1, 1,000-step budget. Preserve the witness setup that shifts ketchup_1 by [0.4, 0, 0]. Replay through call 44:

{"tool":"pi0_pick","input":{"prompt":"pick up the ketchup bottle","max_chunks":20,"lift_thresh":0.05,"gripper_closed_thresh":0.06}}

The historical result is success=true, peak_lift_m=0.0526, and gripper opening approximately 0.0022 m, although neither finger contacts the object. The ketchup moves only about 0.06 cm during the following 150 steps. The planner later interprets the missing object as a dropped grasp.

Expected behavior and validation

Separate a motion-based early-stop condition from verified grasp status in the structured result, as the prompt already does in prose. If contact evidence is intentionally unavailable, expose the heuristic as such and represent grasp status as unknown rather than implying verified pickup. Any proposed simulator contact API must respect the integration's perception-isolation policy; do not silently expose privileged object state. If contact-based reporting is permitted and added, define its object domain and cover empty grasps, large objects, fixtures, and episode-end paths. Contact alone should not be described as a guarantee of stable retention.

Activity

  1. akushonkamen commented on Oct 11, 2026

    @akushonkamen

    Confirmed — I can reproduce the decision rule locally against main (a8c5311).

    Root cause (robots/libero/tools.py):

    • tools.py:213 — gripper_open_thresh: float = 0.0, so the lower bound of the "closed" window is inert by default.
    • tools.py:256 — closed = gripper_open_thresh <= grip < gripper_closed_thresh marks any opening below gripper_closed_thresh (e.g. 0.002 m closing on air) as closed.
    • tools.py:257-259 — success = descent_done and ascended and closed with no contact/held-object evidence, contradicting the prose contract in robots/libero/prompts/evaluate.py:209-219 (Rule 1b: success is "a HINT, not proof").

    Plan: keep the kinematic early-stop heuristic but add an honest grasp_status field (defaulting to "unknown" — no privileged state exposure, no behavior change to success), rewrite the success docstring as "motion heuristic only", and align the evaluate.py Rule 1b wording. Offline unit tests following the issue's probe pattern (VLA/env boundary stubbed) in tests/unit_tests/robots/libero/.

    I'm using an AI coding assistant for this fix; all code and tests are reviewed by me.

  2. akushonkamen commented on Oct 11, 2026

    @akushonkamen

    Opened PR #266 with the fix (root cause and scope as detailed in my claim comment above).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions