Maybe I’m being dumb, but I feel like spelling out some of your ideas would have been useful. (Or maybe you’re just playing with ~pre-rigor intuitions, and I’m overthinking this.)
I think “float to the top” could plausibly mean:
A. In practice, human nature biases us towards treating these ideas as if they were true.
B. Ideal reasoning implies that these ideas should be treated as if they were true.
C. By postulate, these ideas end up reaching fixation in society. [Which then implies things about what members of society can and can’t recognize, e.g. the existence of AIXI-like actors.]
Likewise, what level do you want a NAT to be implemented at? Personal behavior? Structure of group blog sites? Social norms?
Likewise, what level do you want a NAT to be implemented at? Personal behavior? Structure of group blog sites? Social norms?
personal behavior: probably not viable without a dystopian regime of microchips embedded into brains.
structure of group blog sites: maybe—these things have been suggested and tried, i.e. I can’t tell you how many times I’ve seen a reddit comment lamenting the incentives of their upvote system.
weirdly, I found out about the Brave browser last week (weird because it’s apparently been around for a while): attempting to overthrow advertising with an attention-measuring coin. This is great news!
I was thinking a lot about NAT reading this paper. In the context of debate judges, NAT is a bit of a “last minute jerry-rig / frantically shore up the levy” solution, something engineers would stumble upon in an elaborate and convoluted debugging process—the exact opposite of the kind of the solutions alignment researchers are interested in.
if an AI comes to you and says, “I would like to design the particle accelerator this way because,” and then makes to you an inscrutable argument about physics, you’re faced with this tough choice. You can either sign off on that decision and see if it has good consequences, or you can be like, no, don’t do that ’cause I don’t understand it.
—Paul Christiano
I liked the playful writing here.
Maybe I’m being dumb, but I feel like spelling out some of your ideas would have been useful. (Or maybe you’re just playing with ~pre-rigor intuitions, and I’m overthinking this.)
I think “float to the top” could plausibly mean:
A. In practice, human nature biases us towards treating these ideas as if they were true.
B. Ideal reasoning implies that these ideas should be treated as if they were true.
C. By postulate, these ideas end up reaching fixation in society. [Which then implies things about what members of society can and can’t recognize, e.g. the existence of AIXI-like actors.]
Likewise, what level do you want a NAT to be implemented at? Personal behavior? Structure of group blog sites? Social norms?
thanks for your comment.
personal behavior: probably not viable without a dystopian regime of microchips embedded into brains.
structure of group blog sites: maybe—these things have been suggested and tried, i.e. I can’t tell you how many times I’ve seen a reddit comment lamenting the incentives of their upvote system.
weirdly, I found out about the Brave browser last week (weird because it’s apparently been around for a while): attempting to overthrow advertising with an attention-measuring coin. This is great news!
I was thinking a lot about NAT reading this paper. In the context of debate judges, NAT is a bit of a “last minute jerry-rig / frantically shore up the levy” solution, something engineers would stumble upon in an elaborate and convoluted debugging process—the exact opposite of the kind of the solutions alignment researchers are interested in.
Tim Wu’s “Is the first amendment obsolete?” is important and I think everybody should read it.