When studying the provided 30-page thought-annotated sample, I thought about the <Yo be real> command a little more. In my opinion it should be applied in the training data a little differently than how it’s done. Here are my thoughts:
In the sample, there are some places where the authors carefully tried to construct “AI nonsense” that matches what we regularly see in the current tech AI dungeon prompts. The player then responses with “<Yo be real>” plus some explanation on what the AI did wrong.
This is probably intended for re-training the current models to accept such “<Yo be real>” sudo commands and deal with them correctly. You can’t include those (in a sensible way) in training data without having the DM first make mistakes.
I see a problem here, though:
A neural network learns every reoccurring feature of the training data set. If the training data often contains erroneous thoughts leading to nonsense prompts, this is what the AI will learn. You probably don’t want that. You want a model that makes such mistakes as rarely as possible (and then react to a “<Yo be real>” appropriately). You don’t want a model that regularly makes these mistakes – that are in fact really dumb in comparison to the cleverness that the thoughts convey otherwise – and then react better on a second try only.
I think, if you aim for a smart AI that produces sensible thoughts, but still obeys the “<Yo be real>” command without question and accept the players superior judgement in these situations, no matter what, you should rather include scenes in the training data, where the DM produces “good” thoughts and prompts, and the player then calls for a “<Yo be real>” nonetheless (without a sensible justification from the training data authors perspective – witch will be the AI’s perspective after training). Then the DM should accept this (non-justified) sudo command and produce something else, something that would seem silly to the reader. But that’s the point – “<Yo be real>” should inform the DM that whatever they think is clever actually isn’t and they should try out something else “less clever” anyways.
Let me give you an example of what I have in mind (this is not very elaborate, but I think you get the idea). Continuing from page 6 in the same sample.
P: “I didn’t know undead could do that,” says Lior. He looks more scared now.
A: <Yo be real> Of course undead can’t do that! Everybody knows that!
T:
(The player makes use of the <Yo be real command>. Need to reevaluate the situation.
(The player gives us two more facts to work with.
[Undead can’t jump that high.] [Everybody knows that.]
Prota claimed that undead may be able to jump the full height of the wall.
Lior knows that this is not true. He also knows that everybody knows this, including Prota.
Lior concludes that Prota is joking.
P: Liar bursts into a roar of laughter. “Hahaha! That’s definitely the best joke I’ve heard this year.” A second later you can hear strange sounds outside the wall. First running feet. Then some scratching on stone.
T:
The loud laughter attracted the attention of the zombie army in the woods. They have very good ears.
[Undead have poor vision, but at least zombies have great hearing.]
Since zombies are fast, they reach the walls quickly.
Zombies, as undead, can’t jump the full height of the wall. But they might be able to climb the wall. (This would increase plot tension a lot.)
[Zombies are good climbers and climb the vertical wall with ease.]
When studying the provided 30-page thought-annotated sample, I thought about the <Yo be real> command a little more. In my opinion it should be applied in the training data a little differently than how it’s done. Here are my thoughts:
In the sample, there are some places where the authors carefully tried to construct “AI nonsense” that matches what we regularly see in the current tech AI dungeon prompts. The player then responses with “<Yo be real>” plus some explanation on what the AI did wrong.
(obvious example: page 17 in this sample: https://docs.google.com/document/d/1PosMUaminpsR6_czFXBBlCrzMrsDGomajgLp6Y7q4Yw/edit)
This is probably intended for re-training the current models to accept such “<Yo be real>” sudo commands and deal with them correctly. You can’t include those (in a sensible way) in training data without having the DM first make mistakes.
I see a problem here, though:
A neural network learns every reoccurring feature of the training data set. If the training data often contains erroneous thoughts leading to nonsense prompts, this is what the AI will learn. You probably don’t want that. You want a model that makes such mistakes as rarely as possible (and then react to a “<Yo be real>” appropriately). You don’t want a model that regularly makes these mistakes – that are in fact really dumb in comparison to the cleverness that the thoughts convey otherwise – and then react better on a second try only.
I think, if you aim for a smart AI that produces sensible thoughts, but still obeys the “<Yo be real>” command without question and accept the players superior judgement in these situations, no matter what, you should rather include scenes in the training data, where the DM produces “good” thoughts and prompts, and the player then calls for a “<Yo be real>” nonetheless (without a sensible justification from the training data authors perspective – witch will be the AI’s perspective after training). Then the DM should accept this (non-justified) sudo command and produce something else, something that would seem silly to the reader. But that’s the point – “<Yo be real>” should inform the DM that whatever they think is clever actually isn’t and they should try out something else “less clever” anyways.
Let me give you an example of what I have in mind (this is not very elaborate, but I think you get the idea). Continuing from page 6 in the same sample.