Why do you believe that “global utility maximization is something that an ASI might independently discover as a worthwhile goal”? (I assume by “utility” you mean something like happiness.)
II.
I’m not sure most people aren’t sadists. Humans have wildly inconsistent personalities in different situations.[1] Few people have even have even noticed their own inconsistencies, fewer still have gone through the process of extracting a coherent set of values from the soup and gradually generalising that set to every context they can think of...
So I wouldn’t be surprised if most of them didn’t just suddenly fancy torture if it’s as easy playing a computer game. I remember several of my classmates torturing fish for fun, and saw what other kids did in GTA San Andreas just because they were curious. While I haven’t been able to find reliable statistics on it, BDSM is super-popular and probably most men score above the minimum on sexual sadism fetishes.
I.
Why do you believe that “global utility maximization is something that an ASI might independently discover as a worthwhile goal”? (I assume by “utility” you mean something like happiness.)
II.
I’m not sure most people aren’t sadists. Humans have wildly inconsistent personalities in different situations.[1] Few people have even have even noticed their own inconsistencies, fewer still have gone through the process of extracting a coherent set of values from the soup and gradually generalising that set to every context they can think of...
So I wouldn’t be surprised if most of them didn’t just suddenly fancy torture if it’s as easy playing a computer game. I remember several of my classmates torturing fish for fun, and saw what other kids did in GTA San Andreas just because they were curious. While I haven’t been able to find reliable statistics on it, BDSM is super-popular and probably most men score above the minimum on sexual sadism fetishes.
Much like ChatGPT has a large library of simulated personalities (“simulacra”) that it samples from to deal with different contexts.