Summary
Three task-perturbation tasks ship pruned_init files with zero initial states. Any loader that picks an init state by index fails on them. As a result, these tasks can't be reset or evaluated. Results on these suites report a denominator of 90 or 80 instead of 100 per suite, which makes them hard to compare.
| Suite |
Task id |
pruned_init file |
Size |
States |
libero_10_task |
2 |
KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it.pruned_init |
364 B |
0 |
libero_spatial_task |
3 |
pick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate.pruned_init |
364 B |
0 |
libero_spatial_task |
7 |
pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate.pruned_init |
364 B |
0 |
Reproduction
With the package installed from any of the branches:
from liberopro.liberopro import benchmark
for suite_name, task_id in [("libero_10_task", 2), ("libero_spatial_task", 3), ("libero_spatial_task", 7)]:
suite = benchmark.get_benchmark_dict()[suite_name]()
print(suite_name, task_id, len(suite.get_task_init_states(task_id)))
# libero_10_task 2 0
# libero_spatial_task 3 0
# libero_spatial_task 7 0
The BDDL files for these tasks look inconsistent too
These may be why every init state got pruned:
libero_10_task/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it.bddl:
the language is "turn on the stove and put the pan on it" and the goal is
(Turnon flat_stove_1) (On chefmate_8_frypan_1 flat_stove_1_cook_region), but
the file name refers to the moka pot.
libero_spatial_task/pick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate.bddl
and .../pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate.bddl:
both have the language "Pick the akita black bowl on the top of the cabinet
and place it on the plate" and the goal (On akita_black_bowl_2 plate_1).
They differ only in where akita_black_bowl_1 starts (cookies_1 vs.
flat_stove_1_cook_region), so the two tasks are effectively duplicates.
Questions
- Can you publish regenerated init states (and
corrected BDDL if needed)?
Summary
Three task-perturbation tasks ship
pruned_initfiles with zero initial states. Any loader that picks an init state by index fails on them. As a result, these tasks can't be reset or evaluated. Results on these suites report a denominator of 90 or 80 instead of 100 per suite, which makes them hard to compare.pruned_initfilelibero_10_taskKITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it.pruned_initlibero_spatial_taskpick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate.pruned_initlibero_spatial_taskpick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate.pruned_initReproduction
With the package installed from any of the branches:
The BDDL files for these tasks look inconsistent too
These may be why every init state got pruned:
libero_10_task/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it.bddl:the language is "turn on the stove and put the pan on it" and the goal is
(Turnon flat_stove_1) (On chefmate_8_frypan_1 flat_stove_1_cook_region), butthe file name refers to the moka pot.
libero_spatial_task/pick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate.bddland
.../pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate.bddl:both have the language "Pick the akita black bowl on the top of the cabinet
and place it on the plate" and the goal
(On akita_black_bowl_2 plate_1).They differ only in where
akita_black_bowl_1starts (cookies_1vs.flat_stove_1_cook_region), so the two tasks are effectively duplicates.Questions
corrected BDDL if needed)?