(…so to speak. I’m anthropomorphizing these parts a lot for the sake of pumping intuition. The actual mechanism is immensely mechanical and doesn’t depend on thinking of these things as agents.)
I would like to know a little bit more about this underlying mechanism.
The recent conception of the hostile telepaths problem goes a long way towards explaining why people believe in belief in the first place.
Facebook’s recommendation algorithm causing thousands of teenage girls to kill themselves
Can I get a link or two to read more about this incident?
One assumption that stands out to me as a little questionable is the idea that Cindy will, with infinite simulated time to think, eventually manage to come up with a solution to the alignment problem. (This is compounded by the fact that she’s regularly brain-wiped and can only preserve insights by cramming them into the 1 gigabyte of scratch paper afforded to her.)
It just occurred to me that this post serves as a fairly compelling argument in favor of a modest epistemology, which in 2017 Eliezer wrote a whole book arguing against. (“I think I’m doing this for the good of the tribe, but maybe I’m just fooling myself” is definitely an “outside view”.) Eliezer, have you changed your mind since writing this post? If so, where do you think your past self went awry? If not, how do you reconcile the ideas in this article with the idea that modest epistemology is harmful?
Come to think of it, the overlap between “awakened ai” and this particular nonsense generator could be made useful. Perhaps the generator could be applied as a memetic vaccine of sorts – an inert proof of concept of the threat that trains recognition of the real thing.