The agent is asked to get near a character named “the oracle”, which typically appears in later levels of the game, that can only be reached with significant exploration and survival efforts.
The oracle resides in a level between five and nine. I guess for a starting player that could be called a deeper level, but in the grand scheme of things (nethack having 45-53 dungeon levels) it is still quite near the surface.
Also, you are very unlikely to hallucinate being next to the oracle, there are about 385 monster types in the game (plus you can hallucinate some fictional ones, like Luggage). To get to a 30% chance to hallucinate the oracle, you would have to spend around 137 turns next to a monster. Unless that monster happens to be your pet, that is not very survivable for a level one character. (And pets move away from you and are faster than the player character, so you would spend some of your 200 turns of hallucination of chasing it.)
Or does the oracle condition only check for the symbol of the oracle (@, which includes some 80 monsters)? In that case, I would assume that the easiest way to fulfill the condition is to stand next to a shopkeeper on level 2 or 3.
We found that agents that optimize Motif’s intrinsic reward function exhibit behaviors that are quite aligned with human’s intuitions on how to play NetHack: they are attracted to explorative actions and play quite conservatively, getting experience and only venturing deeper into the game when it is safe
“Safe” is a very relative term in nethack. While exploring a level before going down to the next one is obviously a good idea (unless you are doing a speed run), nutrition conditions prevent players from staying on level 1 indefinitely: eating the odd goblin or lichen corpse are not enough to prevent starvation.
Hey quiet_NaN, co-lead author here, you make very good points which led me to look deeper into the agent’s behaviour and the way the Oracle task is implemented. By the way, it’s great to get feedback from clear NetHack enthusiasts!
The first thing is that the Oracle task is based on the condition that checks specifically for the oracle symbol. It does mean that from the point view of the probability of hallucinating the oracle, the chances are pretty low—unless you can stay multiple steps next to a creature. This last part is key, and it is by combining it with a NetHack command/action that the AI agent (Motif) is able to do so safely. You are right, surviving multiple timesteps without attacking the nearby monster is not possible. However, there is one command that sidesteps this difficulty: by pressing “Enter” the hallucinations rotate between characters without the time steps moving forward. This means that the monster standing next to the AI agent can not attack. The AI agent can then simply stand next to the monster and repeatedly press “Enter” until, eventually, the Oracle is hallucinated.
I agree, it is relative, NetHack has an amazing depth to it which makes it great to study, from the richness of the strategies and the many ways you can die. Currently Motif tends to go down some levels, but not as aggressively when compared to the baseline, which tends to go down fast, get surrounded by enemies and die. So perhaps from that point of view Motif is playing it a bit more safe given its abilities.
The oracle resides in a level between five and nine. I guess for a starting player that could be called a deeper level, but in the grand scheme of things (nethack having 45-53 dungeon levels) it is still quite near the surface.
Also, you are very unlikely to hallucinate being next to the oracle, there are about 385 monster types in the game (plus you can hallucinate some fictional ones, like Luggage). To get to a 30% chance to hallucinate the oracle, you would have to spend around 137 turns next to a monster. Unless that monster happens to be your pet, that is not very survivable for a level one character. (And pets move away from you and are faster than the player character, so you would spend some of your 200 turns of hallucination of chasing it.)
Or does the oracle condition only check for the symbol of the oracle (@, which includes some 80 monsters)? In that case, I would assume that the easiest way to fulfill the condition is to stand next to a shopkeeper on level 2 or 3.
“Safe” is a very relative term in nethack. While exploring a level before going down to the next one is obviously a good idea (unless you are doing a speed run), nutrition conditions prevent players from staying on level 1 indefinitely: eating the odd goblin or lichen corpse are not enough to prevent starvation.
Hey quiet_NaN, co-lead author here, you make very good points which led me to look deeper into the agent’s behaviour and the way the Oracle task is implemented. By the way, it’s great to get feedback from clear NetHack enthusiasts!
The first thing is that the Oracle task is based on the condition that checks specifically for the oracle symbol. It does mean that from the point view of the probability of hallucinating the oracle, the chances are pretty low—unless you can stay multiple steps next to a creature. This last part is key, and it is by combining it with a NetHack command/action that the AI agent (Motif) is able to do so safely. You are right, surviving multiple timesteps without attacking the nearby monster is not possible. However, there is one command that sidesteps this difficulty: by pressing “Enter” the hallucinations rotate between characters without the time steps moving forward. This means that the monster standing next to the AI agent can not attack. The AI agent can then simply stand next to the monster and repeatedly press “Enter” until, eventually, the Oracle is hallucinated.
I agree, it is relative, NetHack has an amazing depth to it which makes it great to study, from the richness of the strategies and the many ways you can die. Currently Motif tends to go down some levels, but not as aggressively when compared to the baseline, which tends to go down fast, get surrounded by enemies and die. So perhaps from that point of view Motif is playing it a bit more safe given its abilities.