Ideas for NLE, Fall/Winter 2021 edition

[heiner]

Ideas for the next NLE

This issue is for collecting ideas for the "next major version" of NLE. I'd like to collect wild ideas, too, if only to dismiss them as unrealistic.

Let's jump right in.

Some random ideas

• Given how we didn't ever finish the menu observations (Add menu_strs observations. #207), the "full game" is currently only available via tty_chars and friends. Perhaps we can make a virtue of necessity and just embrace that? In the strongest sense this might mean: Remove glyphs, chars, colors, message, inv_*, screen_descriptions, and perhaps a couple of the others. (Note that this change is going into the opposite direction than feature requests like Added pixel-based observations using "pixel" observation_keys #279.)
• blstats would probably have to stay as it's used in defining tasks.
• However, many of our original tasks were somewhat ad hoc and are perhaps no longer that useful. This includes staircase, staircasepet, oracle, gold, eat, and scout. If we go the "maximal disruption" route, we could consider removing these tasks.
• If we were to remove these tasks, we could potentially provide a purely C++-backed versions of "score" and "ascension" (with reward +1 for ascending, -1 for dying, and perhaps 0 for escaping? :D). Given that by some calculations, 90% of the time running NLE is spend in Python layers, this might give us another 10x speed increase. The cost is a less configurable environment. However, if we keep, say, blstats around, wrappers a la the current task definitions would still be possible.

Independently of the above, there is the "breaking changes" label where we gather some changes we've queued up for after the competition.

[cdmatters]

Perf Ideas

feel free to edit

• remove one memcopy per step for about all but tty_* observations (glyphs, colors, ... , inv_*, ...,screen_descriptions) - NB observation keys is only glyphs its already 2kB memcopy per step.
• move reward fn to C++
• add mechanism to set time OR randomise the start time. This has important experimental/reproducibility value.