Include "Pop-up" windows in observation

[paulkent-um]

🚀 Feature

Include pop-up windows (such as "Things that are here:", or the spellcasting menu) in the observation space, either under "message", or under a new key.

Motivation

I'm developing a bot for the 2021 Nethack challenge. The prospect of having to figure out where in the raw terminal output pop-ups are drawn up is not pleasing. Currently, if I want to get the information in these pop-up windows, that is the only way.

Pitch

Copy over the information in the pop-ups under the observation key "message".

Alternatives

If the pop-up windows turn out to take more than 255 characters to convey in some cases, or if it's possible for a message and a pop-up window to appear simultaneously, you could instead add a new key for this purpose, and allot it a larger buffer than the normal message observation.

[cdmatters]

Hi there @paulkent-um - I'm afraid copying over the item to message is a no go for a number of reasons:

• Firstly the message buffer would almost certainly be too short and we would have to cut the popups. Implementing our own buffering system is also a no go, since it would break a pretty fundamental design that whatever keypress you send goes to the game. It would be a little crazy, and involve hacks throughout the stack.
• Secondly, menus are more complicated that just messages. There's selections that take place ('what to drop') and associated letters/shortcuts, and more. Perhaps for your use case parsing a giant string might suit you, but be certain that it would suddenly lead to edge cases. Consider a dropping menu: When choosing what to drop, NH gives you sub headings as below:

These are rendered as spearate rows, some are selectable, some arent. I have no doubt this is not the ideal solution for you either.

I have opened up a draft pr #207, with a proposed solution, with a new observation key, and after some discussions I'll revert with decisions on how to proceed.

[paulkent-um]

I've left a comment on #207. One thing I did want to say here is:

These are rendered as spearate rows, some are selectable, some arent. I have no doubt this is not the ideal solution for you either.

Perhaps not ideal, but if it was up to me, I'd settle for it. It'd take some unpacking to make useful, but I could absolutely work with that, which is a lot more than I can say about the present version of NLE.