I think that ordinarily, social manipulation games do not erode the norm of being able to have honest conversations. I think you are on some level aware of the norms I’m going to describe and are acting in accordance with them, I just want to describe them explicitly. As I understand them, the norms for playing social manipulation games is that there is a distinction between statements made with the game and statements made outside the game. Statements made within the game are not bound by the norms of honesty outside of the game. A player lying or misleading within a game does not impact their reputation outside the game, tho it may impact a reputation a playing is trying to maintain within the game that other players are tracking separately. There is an implicit agreement by joining a social manipulation game to suspend these rules of honesty.
A difficulty is that like all implicit agreements the details of it can be misunderstood; in particular, it can ambiguous which statements are made within the game and outside the game. Certainly not every statement within the duration of the game is within the game in this sense — if a player says “I have an important appointment so I need to stop playing soon” and is lying then that would be a genuine norm-violating dishonesty. In casual game-playing people often discuss the strategy of the game during the game and that can be considered statements outside the game. This is especially true for a game like Settlers of Catan which is not primarily a social manipulation game, tho it involves some social strategizing. If a false or misleading statement is made which some people think is within the game and others think is outside the game then that does lead to a genuine degradation of the norm of honesty.
One way to remedy this is to aim to be more explicit about the norms of the game before playing. For example, in your case, asking the other player whether they agree to not trick or target people. If they disagree, the explictness has nonetheless deescalated the dispute from a challenge to personal integrity into a disagreement on how to play games. This disagreement can still be acrimonious, for example leading the two of you not to play games together, but I think it’s an improvement.
AI Interpretability idea: Train an LLM with a “split-brain” architecture. That is, the data in the intermediate layers is siloed into two groups with only a small amount of interaction between the groups. This could be enforced in the linear layers by forcing the cross interaction to be a low-rank matrix, or by adding to the regularizer some matrix norm of the cross interaction matrices. I’m sure this can be generalized from MLP to transformer architecture.
Next, apply standard interpretability tools to see how it uses its two sides to represent concepts. Hopefully you’ll find that there are some concepts and tasks that are more focused on one side than on another.
Next, try to elicit the LLM to give verbal descriptions of which concept is represented in which side. Easy mode is to explicitly tell it how it is designed, give some examples of specific concepts primarily encoded on specific sides, even fine-tune train it with those examples, and ask it to predict in which sides are concepts you haven’t trained/told it about before. Hard mode is not to tell it anything about how it was designed but just give it general prompts about trying to introspect try to get it to figure this out from scratch. Maybe hand it off to some of the LLM whisperers without telling it how it was created.
Finally, if you succeed in hard mode, give the same prompting to a regular LLM and pay attention to what it says.
The idea is that I’m trying to elicit the LLM to introspect in a way that actually gives us information about the features and concepts it uses to generate text, rather than just tell the story of the humanoid character it is playacting as. I want to create a scenario where there is a non-obvious ground truth that we have access to that it could plausibly have access to and be able to share.