Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Default metadata in BaseMujocoEnv #1099

Closed
wants to merge 9 commits into from

Conversation

spiglerg
Copy link

Description

The current code for BaseMujocoEnv requires the env metadata dictionary to have fixed, pre-specified values. While this may be useful for future API changes, it doesn't seem very useful at the moment.

Fixes #1059

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing unit tests pass locally with my changes

@Kallinteris-Andreas
Copy link
Collaborator

i will be inactive till 11/07/2024, after which i will review it

@@ -17,7 +17,7 @@ class AntEnv(MujocoEnv, utils.EzPickle):
"rgb_array",
"depth_array",
],
"render_fps": 20,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this changed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I calculate that dynamically in the constructor, since it depends on self.dt and before it was forcing it to always have this value anyway:

Previous code:

assert (
int(np.round(1.0 / self.dt)) == self.metadata["render_fps"]
), f'Expected value: {int(np.round(1.0 / self.dt))}, Actual value: {self.metadata["render_fps"]}'

assert (
int(np.round(1.0 / self.dt)) == self.metadata["render_fps"]
), f'Expected value: {int(np.round(1.0 / self.dt))}, Actual value: {self.metadata["render_fps"]}'
if "render_fps" not in self.metadata:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the functionality for verifying that render_fps equals the expected value is removed (for v4 or older based environments that define the render_fps themselfs)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm right, but if render_fps is in metadata and is different than int(np.round(1.0 / self.dt)), the assertion fails. We can maintain the previous behavior by always overriding render_fps with the computed value for the moment then, perhaps?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it should fail if render_fps is explicitly defined to something else, otherwise render issues will arise

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm ok!
Then I can modify it to set this computed value if render_fps is not in the metadata, and run the assertion if it is?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I also update the individual Mujoco environments to remove the metadata -where it matches the new automatically generated one- ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leave all the v4 and older environment as is

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok!

@Kallinteris-Andreas
Copy link
Collaborator

From my testing, everything appears to work

import gymnasium
from gymnasium.envs.mujoco.mujoco_env import MujocoEnv
assert MujocoEnv.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array']}

env0 = gymnasium.make("Ant-v5")
assert MujocoEnv.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array']}
assert env0.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 20}

env1 = gymnasium.make("Humanoid-v5")
assert MujocoEnv.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array']}
assert env0.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 20}
assert env1.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 67}

env2 = gymnasium.make("Hopper-v4")
assert MujocoEnv.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array']}
assert env0.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 20}
assert env1.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 67}
assert env2.metadata == {'render_modes': ['human', 'rgb_array', 'depth_array'], "render_fps": 125}

@spiglerg
Copy link
Author

Done. :)

@Kallinteris-Andreas
Copy link
Collaborator

run pre-commit run --all-files check contribute.md for clarification

@spiglerg
Copy link
Author

There was a problem with a line in the file not committed. Now pre-commit tests run locally.

@spiglerg
Copy link
Author

Apologies; the last problem was due to `frames_skip' afftecting self.dt. I am fixing and running tests locally. I will push a fix as soon as possible.

if "render_fps" in self.metadata:
assert (
int(np.round(1.0 / self.dt)) == self.metadata["render_fps"]
), f'Expected value: {int(np.round(1.0 / self.dt))}, Actual value: {self.metadata["render_fps"]}'
else:
# Make a copy of the dictionary to avoid modifying the class variable
self.metadata = self.metadata.copy()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we need, wouldn't be easier to do

    metadata = {
        "render_modes": [
            "human",
            "rgb_array",
            "depth_array",
        ],
    "render_fps": int(np.round(1.0 / self.dt))
    }
    ```

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I am not sure 'self.dt' can be called from there, as `metadata' would be a static class variable?

That is also why problems arise if metadata' is not copied before replacing values (in the new code) -> if derived classes don't specifically override metadata', the same (possibly wrong, in case of `render_fps') values end up shared between them.

Copy link
Collaborator

@Kallinteris-Andreas Kallinteris-Andreas Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry meant to say, there is no need to have a metadata static class variable

    self.metadata = {
        "render_modes": [
            "human",
            "rgb_array",
            "depth_array",
        ],
    "render_fps": int(np.round(1.0 / self.dt))
    }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was my original intention. ;)
However, if metadata is initialized as self.metadata in the constructor (as in this approach), but a derived class still implements the older way with metadata as static class variable, the outcome may be unexpected for the user (hence the use of assertions in the latest code, in case for some reason a mismatch happens).

I can test that and add a pytest to check for correct behavior in case a derived class re-defines the metadata as static (sub)class variable, as is currently done.
In any case, the current main repository does not allow for any derived class to have any different metadata, so any metadata written explicitly by a user (for a new derived class) must be identical to this value).

It would also help to remove all 'metadata' from the existing Mujoco environments to improve clarity, and indeed only define it once in the base class. Or perhaps to change them from class variables to object variables initialized in the constructor (i.e. "render_fps" can be always removed, since there is only one valid value; "render_modes" can be set to replace the default array, e.g.

self.metadata["render_modes"] = [new-modes]

eploiting the fact that the default metadata would always contain the two keys, "render_modes" and "render_fps").

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

** This applying to MujocoEnv and derived classes.
gym.Env also (ab)uses static class variables, including metadata (although it is not strict like MujocoEnv in the choice of "render_fps") ?

In that case, if metadata is kept a class variable as in gym.Env, then dynamic copying (metadata.copy()) and updating of "render_fps" the constructor may be the only alternative.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, "render_fps" is only used either explicitly by a few environments that handle their own rendering, or as argument to be explicitly passed by the user to the `save_video' utility function.

It does not seem to be a problem to just set the correct value (for Mujoco envs) without checks on later modifications by the users. Perhaps we can add a comment in the code to explain that Mujoco environments should leave the default "render_fps" as is, since there is no use in ever modifying it.

@pseudo-rnd-thoughts
Copy link
Member

@Kallinteris-Andreas What is happening with this PR?

@Kallinteris-Andreas
Copy link
Collaborator

I am closing as the "issue" is very minor, and the solution may introduce issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Proposal] Default metadata in BaseMujocoEnv
3 participants