Skip to content

serialize_observation() silently drops Observation.metadata #616

@goodmorningsaksham

Description

@goodmorningsaksham

Bug

serialize_observation() in env_server/serialization.py excludes metadata from the serialized output alongside reward and done:

obs_dict = observation.model_dump(
    exclude={"reward", "done", "metadata"}
)

reward and done are intentionally excluded because they’re promoted to top-level sibling keys in the response. But metadata is excluded and never re-added, it’s silently lost.

Any data an environment author puts in Observation.metadata (step counts, task progress, debug info, custom metrics) vanishes when the observation crosses the wire via WebSocket or HTTP.

How I found this

I built https://huggingface.co/spaces/saksham1771/infinite-dom
A procedurally generated web environment for training browser agents, as my submission for the OpenEnv Hackathon India 2026.

Our environment returns task progress metadata (total graph nodes, completed checkpoints) via Observation.metadata. During live evaluation over WebSocket, I noticed the metadata was silently absent on the client side.

I spent a good amount of debugging time adding multiple fallback paths before I traced the data loss to this specific line in serialization.py.

What confirmed it as a bug

Action.metadata survives serialization fine (client + server) because _step_payload() calls model_dump() without excluding metadata.

So metadata flows in one direction but not the other. That asymmetry doesn't look intentional.

Reproduction

from openenv.core.env_server.serialization import serialize_observation
from openenv.core.env_server.types import Observation

obs = Observation(
    done=False,
    reward=0.5,
    metadata={"total_nodes": 5, "task_id": 1},
)

result = serialize_observation(obs)

print(result)
# {'observation': {}, 'reward': 0.5, 'done': False}
# ^ metadata is gone; expected it to be here

Expected behavior

metadata should be promoted to a top-level key in the response, consistent with how reward and done are handled:

{
    "observation": {},
    "reward": 0.5,
    "done": False,
    "metadata": {"total_nodes": 5, "task_id": 1}
}

I have a fix ready with tests , opening a PR shortly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions