[Question] generated demo looks weird

[Henry0517code]

Question

Hi, I’m trying to generate demonstrations from my dataset, but the results are strange and inconsistent.

During the generation process, I observed many unexpected behaviors — for example, the robot arm suddenly raises or moves unpredictably.
Even after tightening the success conditions, the generated trajectories are still not usable and not as good as the official Franka Stack example.

I attempted the following adjustments:

• Reduced noise and randomization parameters in MimicEnvCfg
• Tweaked task configuration and thresholds

I’m now suspecting that my dataset was collected using JointPositionActionCfg, which might be incompatible with the IK-based mimic environment

Any insights or suggestions would be greatly appreciated — thank you!

Dataset

Screencast.from.10-28-2025.04.33.16.PM.webm

Generate demo

Screencast.from.10-28-2025.04.35.51.PM.webm

Generation process

Screencast.from.10-28-2025.04.39.40.PM.webm

Implementation details

MimicEncCfg

from isaaclab.managers import ObservationGroupCfg as ObsGroup
from isaaclab.managers import ObservationTermCfg as ObsTerm
from isaaclab.managers import SceneEntityCfg
from isaaclab.envs.mimic_env_cfg import MimicEnvCfg, SubTaskConfig
from isaaclab.utils import configclass

from ramen.tasks.manager_based import mdp
from .joint_pos_env_cfg import EggPickPlaceEnvCfg
from ramen.tasks.manager_based.ramen_env_cfg import ObservationsCfg as ObservationsCfgTemplate


@configclass
class ObservationsCfg(ObservationsCfgTemplate):
    """Configuration for observations with subtask definitions"""

    @configclass
    class SubtaskCfg(ObsGroup):
        """Observations for subtask group."""

        pick_egg = ObsTerm(
            func=mdp.object_grasped,
            params={
                "robot_cfg": SceneEntityCfg("robot"),
                "ee_frame_cfg": SceneEntityCfg("ee_frame"),
                "object_cfg": SceneEntityCfg("egg"),
                "gripper_joint_names": ["left_inner_knuckle_joint", "right_inner_knuckle_joint"],
                "lift_threshold": 0.9
            },
        )

        def __post_init__(self):
            self.enable_corruption = False
            self.concatenate_terms = False

    subtask_terms: SubtaskCfg = SubtaskCfg()
    def __post_init__(self):
        # post init of parent
        super().__post_init__()


@configclass
class EggPickPlaceMimicEnvCfg(EggPickPlaceEnvCfg, MimicEnvCfg):
    """
    Isaac Lab Mimic environment config class.
    """

    def __post_init__(self):
        # post init of parents
        super().__post_init__()

        # add subtask obs
        self.observations = ObservationsCfg()

        # Override the existing values
        self.datagen_config.name = "egg pickplace mimic"
        self.datagen_config.generation_guarantee = True
        self.datagen_config.generation_keep_failed = True
        self.datagen_config.generation_num_trials = 10
        self.datagen_config.generation_select_src_per_subtask = True
        self.datagen_config.generation_transform_first_robot_pose = False
        self.datagen_config.generation_interpolate_from_last_target_pose = True
        self.datagen_config.generation_relative = True
        self.datagen_config.max_num_failures = 25
        self.datagen_config.seed = 1

        # The following are the subtask configurations for the task.
        subtask_configs = []
        subtask_configs.append(
            SubTaskConfig(
                # Each subtask involves manipulation with respect to a single object frame.
                object_ref="egg",
                # This key corresponds to the binary indicator in "datagen_info" that signals
                # when this subtask is finished (e.g., on a 0 to 1 edge).
                subtask_term_signal="pick_egg",
                # Specifies time offsets for data generation when splitting a trajectory into
                # subtask segments. Random offsets are added to the termination boundary.
                subtask_term_offset_range=(5, 10),
                # Selection strategy for the source subtask segment during data generation
                selection_strategy="nearest_neighbor_object",
                # Optional parameters for the selection strategy function
                selection_strategy_kwargs={"nn_k": 3},
                # Amount of action noise to apply during this subtask
                action_noise=0.0,
                # Number of interpolation steps to bridge to this subtask segment
                num_interpolation_steps=50,
                # Additional fixed steps for the robot to reach the necessary pose
                num_fixed_steps=0,
                # If True, apply action noise during the interpolation phase and execution
                apply_noise_during_interpolation=False,
                description="Pick up the egg from the plate",
                next_subtask_description="Place the egg into the bowl",
            )
        )

        subtask_configs.append(
            SubTaskConfig(
                # Each subtask involves manipulation with respect to a single object frame.
                object_ref="noodle_bowl",
                # End of final subtask does not need to be detected
                subtask_term_signal=None,
                # No time offsets for the final subtask
                subtask_term_offset_range=(0, 0),
                # Selection strategy for source subtask segment
                selection_strategy="nearest_neighbor_object",
                # Optional parameters for the selection strategy function
                selection_strategy_kwargs={"nn_k": 3},
                # Amount of action noise to apply during this subtask
                action_noise=0.0,
                # Number of interpolation steps to bridge to this subtask segment
                num_interpolation_steps=50,
                # Additional fixed steps for the robot to reach the necessary pose
                num_fixed_steps=0,
                # If True, apply action noise during the interpolation phase and execution
                apply_noise_during_interpolation=False,
            )
        )

        self.subtask_configs["ur3e"] = subtask_configs

MimicEnv

import torch
from collections.abc import Sequence

import isaaclab.utils.math as PoseUtils
from isaaclab.envs import ManagerBasedRLMimicEnv

class EggPickPlaceMimicEnv(ManagerBasedRLMimicEnv):
    """
    Isaac Lab Mimic environment wrapper class.
    """

    def get_robot_eef_pose(self, eef_name: str, env_ids: Sequence[int] | None = None) -> torch.Tensor:
        """
        Get current robot end effector pose. Should be the same frame as used by the robot end-effector controller.

        Args:
            eef_name: Name of the end effector.
            env_ids: Environment indices to get the pose for. If None, all envs are considered.

        Returns:
            A torch.Tensor eef pose matrix. Shape is (len(env_ids), 4, 4)
        """
        if env_ids is None:
            env_ids = slice(None)

        # Retrieve end effector pose from the observation buffer
        eef_pos = self.obs_buf["policy"]["eef_pos"][env_ids]
        eef_quat = self.obs_buf["policy"]["eef_quat"][env_ids]
        # Quaternion format is w,x,y,z
        return PoseUtils.make_pose(eef_pos, PoseUtils.matrix_from_quat(eef_quat))
    
    def target_eef_pose_to_action(
        self,
        target_eef_pose_dict: dict,
        gripper_action_dict: dict,
        action_noise_dict: dict | None = None,
        env_id: int = 0,
    ) -> torch.Tensor:
        """
        Takes a target pose and gripper action for the end effector controller and returns an action
        (usually a normalized delta pose action) to try and achieve that target pose.
        Noise is added to the target pose action if specified.

        Args:
            target_eef_pose_dict: Dictionary of 4x4 target eef pose for each end-effector.
            gripper_action_dict: Dictionary of gripper actions for each end-effector.
            noise: Noise to add to the action. If None, no noise is added.
            env_id: Environment index to get the action for.

        Returns:
            An action torch.Tensor that's compatible with env.step().
        """
        eef_name = list(self.cfg.subtask_configs.keys())[0]

        # target position and rotation
        (target_eef_pose,) = target_eef_pose_dict.values()
        target_pos, target_rot = PoseUtils.unmake_pose(target_eef_pose)

        # current position and rotation
        curr_pose = self.get_robot_eef_pose(eef_name, env_ids=[env_id])[0]
        curr_pos, curr_rot = PoseUtils.unmake_pose(curr_pose)

        # normalized delta position action
        delta_position = target_pos - curr_pos

        # normalized delta rotation action
        delta_rot_mat = target_rot.matmul(curr_rot.transpose(-1, -2))
        delta_quat = PoseUtils.quat_from_matrix(delta_rot_mat)
        delta_rotation = PoseUtils.axis_angle_from_quat(delta_quat)

        # get gripper action for single eef
        (gripper_action,) = gripper_action_dict.values()

        # add noise to action
        pose_action = torch.cat([delta_position, delta_rotation], dim=0)
        if action_noise_dict is not None:
            noise = action_noise_dict[eef_name] * torch.randn_like(pose_action)
            pose_action += noise
            pose_action = torch.clamp(pose_action, -1.0, 1.0)

        return torch.cat([pose_action, gripper_action], dim=0)
    
    def action_to_target_eef_pose(self, action: torch.Tensor) -> dict[str, torch.Tensor]:
        """
        Converts action (compatible with env.step) to a target pose for the end effector controller.
        Inverse of @target_eef_pose_to_action. Usually used to infer a sequence of target controller poses
        from a demonstration trajectory using the recorded actions.

        Args:
            action: Environment action. Shape is (num_envs, action_dim)

        Returns:
            A dictionary of eef pose torch.Tensor that @action corresponds to
        """
        eef_name = list(self.cfg.subtask_configs.keys())[0]

        delta_position = action[:, :3]
        delta_rotation = action[:, 3:6]

        # current position and rotation
        curr_pose = self.get_robot_eef_pose(eef_name, env_ids=None)
        curr_pos, curr_rot = PoseUtils.unmake_pose(curr_pose)

        # get pose target
        target_pos = curr_pos + delta_position

        # Convert delta_rotation to axis angle form
        delta_rotation_angle = torch.linalg.norm(delta_rotation, dim=-1, keepdim=True)
        delta_rotation_axis = delta_rotation / delta_rotation_angle

        # Handle invalid division for the case when delta_rotation_angle is close to zero
        is_close_to_zero_angle = torch.isclose(delta_rotation_angle, torch.zeros_like(delta_rotation_angle)).squeeze(1)
        delta_rotation_axis[is_close_to_zero_angle] = torch.zeros_like(delta_rotation_axis)[is_close_to_zero_angle]

        delta_quat = PoseUtils.quat_from_angle_axis(delta_rotation_angle.squeeze(1), delta_rotation_axis).squeeze(0)
        delta_rot_mat = PoseUtils.matrix_from_quat(delta_quat)
        target_rot = torch.matmul(delta_rot_mat, curr_rot)

        target_poses = PoseUtils.make_pose(target_pos, target_rot).clone()

        return {eef_name: target_poses}
    
    def actions_to_gripper_actions(self, actions: torch.Tensor) -> dict[str, torch.Tensor]:
        """
        Extracts the gripper actuation part from a sequence of env actions (compatible with env.step).

        Args:
            actions: environment actions. The shape is (num_envs, num steps in a demo, action_dim).

        Returns:
            A dictionary of torch.Tensor gripper actions. Key to each dict is an eef_name.
        """
        # last dimension is gripper action
        return {list(self.cfg.subtask_configs.keys())[0]: actions[:, -1:]}
    
    def get_subtask_term_signals(self, env_ids: Sequence[int] | None = None) -> dict[str, torch.Tensor]:
        """
        Gets a dictionary of termination signal flags for each subtask in a task. The flag is 1
        when the subtask has been completed and 0 otherwise. The implementation of this method is
        required if intending to enable automatic subtask term signal annotation when running the
        dataset annotation tool. This method can be kept unimplemented if intending to use manual
        subtask term signal annotation.

        Args:
            env_ids: Environment indices to get the termination signals for. If None, all envs are considered.

        Returns:
            A dictionary termination signal flags (False or True) for each subtask.
        """
        if env_ids is None:
            env_ids = slice(None)

        signals = dict()
        subtask_terms = self.obs_buf["subtask_terms"]
        signals["pick_egg"] = subtask_terms["pick_egg"][env_ids]

        # final subtask is placing cubeC on cubeA (motion relative to cubeA) - but final subtask signal is not needed
        return signals

[RandomOakForest]

Thank you for posting this. I'll move the post to our Discussions for follow up. Here is a summary to consider that may be of help.

Your issue likely stems from a mismatch between the action representation used during dataset collection (JointPositionActionCfg) and the IK-based (inverse kinematics) control scheme in your current mimic environment. This can lead to strange behaviors and poor transferability of demonstrations for several technical reasons.

Core Problem: Action Space Mismatch

• JointPositionActionCfg records raw joint positions as actions.
• IK-based MimicEnv expects actions in the end-effector (Cartesian) space and internally computes joint motions using inverse kinematics.
• Attempting to replay joint-space trajectories directly in an environment configured for IK-based mimicry can cause non-physical, erratic, or unstable robot motions, because the way actions are interpreted (and the way the robot moves as a result) are fundamentally different.

Why This Causes Trouble

• The IK controller tries to map incoming actions to a target pose in workspace, not specific joint targets.
• If your replay or mimic code expects workspace goals but is fed joint positions, the resulting movements will not correspond to the intended demonstration — leading to unpredictable or useless behaviors.
• Tightening thresholds or reducing noise/randomization only improves fidelity if the underlying action format matches the environment's expectations.

Steps for Consistent Demonstrations

Solution 1: Match Action Spaces

• Ensure that the dataset and the mimic environment use the same action configuration (either both JointPositionActionCfg or both Cartesian/IK-based ActionCfg).
• If you want to use demonstrations collected using JointPositionActionCfg, recreate your mimic environment with the same ActionCfg instead of the IK variant.
• Alternatively, collect new demonstrations using the IK-based ActionCfg to match your current mimic setup.

Solution 2: Data Conversion

• If you must use existing joint-space trajectories in an IK-based environment, a reliable solution is to convert joint-space actions to the equivalent end-effector (Cartesian) trajectories by forward kinematics, then use those as targets for the IK controller. This requires:
  • Computing the forward kinematics for each set of joint positions in your dataset.
  • Using those end-effector poses as demonstration targets in your mimic environment.
• This step is more challenging and requires careful implementation to avoid artifacts.