I don’t think the former is free from Goodharting either. My sense of a good community is one where we get the character judgment out of the way from the start. So indeed “people evaluating one another on fuzzy personal criteria”. In the sense of “hey we like you for the things about you you can’t change even if you tried”. So personal value is secured, meaning that the person can actually start to pursue the things they value truly for their own sake.
As I said in other places: If I got all of my needs out of the way, I would still work on AI Safety (which I value for it’s own sake), and have a lot more cognitive bandwidth to allocate to it too. All of which is now going to securing my worth. Which is essentially Goodharting, since I’m incentivized to skew everything I do to things that can be easily used for signaling.
An unsatisfied satisficer is a maximizer. I’m maximizing my status, and the useful work I’m doing is only a side effect. That doesn’t seem like a good thing. Especially with a security mindset.
Hm.
I don’t think the former is free from Goodharting either. My sense of a good community is one where we get the character judgment out of the way from the start. So indeed “people evaluating one another on fuzzy personal criteria”. In the sense of “hey we like you for the things about you you can’t change even if you tried”. So personal value is secured, meaning that the person can actually start to pursue the things they value truly for their own sake.
As I said in other places: If I got all of my needs out of the way, I would still work on AI Safety (which I value for it’s own sake), and have a lot more cognitive bandwidth to allocate to it too. All of which is now going to securing my worth. Which is essentially Goodharting, since I’m incentivized to skew everything I do to things that can be easily used for signaling.
An unsatisfied satisficer is a maximizer. I’m maximizing my status, and the useful work I’m doing is only a side effect. That doesn’t seem like a good thing. Especially with a security mindset.