Working on multi-agent safety and cooperative AI. Other interests include s-risks, safe-pareto improvement, and theoretical computer science.
Thao Pham
Karma: 16
A few things I like about Coup are (1) you can learn to make a coalition, or multiple (Coup generally behaves like a zero-sum game, but with 3+ players, coalition is a smart thing to do), (2) analyze the coalition values and learn to break away from coalition when needed (this is not the same as backstabbing, since any temporary coalitions in Coup require players to eventually compete with each other).
One idea for the Team version is to have a general rule like “either member of the team survives, both teammates win” so people can learn to use the cards that are more valuable (I tend to notice that I win more games with a safe combination like “Duke + Ambassador/Captain”, or “Captain + Contessa”).
I used to do translation, and I found vocabulary landscape quite trivial. I can understand both languages fluently but can’t find some equivalent characterization without losing a little bit of their semantic properties.
I feel the same struggle to tackle the Beetle problem. But I also think sometimes it’s fair to not try to force a perfect structural mapping between our internal language and theirs (which is circular if both are private). Likewise, if both beetles are genuinely incompatible, a thoughtful approximation is fine.
I think these advice can also be applied to adults, especially young adults, when many of them are in a transition phase, whose past mistakes might not be too deep or problematic to recover from, or their life principles are still forming.
Personally, I like number #8, specifically:
“Start planning the life you want by thinking freely in your own head. You can beat others by starting earlier because you respect yourself and haven’t fallen for the “children aren’t people”-style propaganda.”
I feel like this is crucial for children who need to overcome adversaries, and even more for those who live in a hostile environment (toxic parents, schools or bully), by setting up priorities and long-term plans, or having agency in general. Though, I do wonder how much of agency a child can cultivate when power is not in their control in the first place.
Thank you for explaining this dynamic model! I’m curious about a few things:
Beyond characterizing “uncooperative” versus “cooperative” behavior, would you also care about “bad cooperativeness” and “good uncooperativeness”?
On the axes of good and bad outcomes, some cooperative behaviors could be harmful (collusion) and some cases, non-cooperation is preferred (misaligned models ignore each other).
“observed misbehaviour will approach the limit of all the misbehaviour that will ever be detected” → Do you assume that the detector is becoming more sensitive without becoming less precise?
I feel like we can’t interpret a rising detection count as rising true misbehavior unless you’re holding FPR fixed, or at least tracking it.