Only a single training example needed through use of hypotheticals.
(to be clear, the question and answer serve less as “training data” meant to represent the user, but as “IDs” or “coordinates” menat to locate the user in past-lightcone.)
We need good inner alignment. (And with this, we also need to understand hypotheticals).
this is true, though i think we might not need a super complex framework for hypotheticals. i have some simple math ideas that i explore a bit here, and about which i might write a bunch more.
for failure modes like the user getting hit by a truck or spilling coffee, we can do things such as at each step asking not 1 cindy the question, but asking 1000 cindy’s 1000 slight variations on the question, and then maybe have some kind of convolutional network to curate their answers (such as ignoring garbled or missing output) and pass them to the next step, without ever relying on a small number of cindy’s except at the very start of this process.
it is true that weird memes could take over the graph of cindy’s; i don’t have an answer to that apart that it seems sufficiently not likely to me that i still think this plan has promise.
Chaos theory. Someone else develops a paperclip maximizer many iterations in, and the paperclip maximizer realizes it’s in a simulation, hacks into the answer channel and returns “make as many paperclips as possible” to the AI.
hmm. that’s possible. i guess i have to hope this never happens on the question-interval, on any simulation day. alternatively, maybe the mutually-checking graph of a 1000 cindy’s can help with this? (but probly not; clippy can just hack the cindy’s).
So all the virtual humans get saved on disk, and then can live in the utopia. Hey, we need loads of people to fill up the dyson sphere anyway.
yup. or, if the QACI user is me, i’m probly also just fine with those local deaths; not a big deal compared to an increased chance of saving the world. alternatively, instead of being saved on disk, they can also just be recomputed later since the whole process is deterministic.
I am not confident that your “make it complicated and personal data” approach at the root really stops all the aliens doing weird acausal stuff.
yup, i’m not confident either. i think there could be other schemes, possibly involving cryptography in some ways, to entangle the answer with a unique randomly generated signature key or something like that.
(to be clear, the question and answer serve less as “training data” meant to represent the user, but as “IDs” or “coordinates” menat to locate the user in past-lightcone.)
this is true, though i think we might not need a super complex framework for hypotheticals. i have some simple math ideas that i explore a bit here, and about which i might write a bunch more.
for failure modes like the user getting hit by a truck or spilling coffee, we can do things such as at each step asking not 1 cindy the question, but asking 1000 cindy’s 1000 slight variations on the question, and then maybe have some kind of convolutional network to curate their answers (such as ignoring garbled or missing output) and pass them to the next step, without ever relying on a small number of cindy’s except at the very start of this process.
it is true that weird memes could take over the graph of cindy’s; i don’t have an answer to that apart that it seems sufficiently not likely to me that i still think this plan has promise.
hmm. that’s possible. i guess i have to hope this never happens on the question-interval, on any simulation day. alternatively, maybe the mutually-checking graph of a 1000 cindy’s can help with this? (but probly not; clippy can just hack the cindy’s).
yup. or, if the QACI user is me, i’m probly also just fine with those local deaths; not a big deal compared to an increased chance of saving the world. alternatively, instead of being saved on disk, they can also just be recomputed later since the whole process is deterministic.
yup, i’m not confident either. i think there could be other schemes, possibly involving cryptography in some ways, to entangle the answer with a unique randomly generated signature key or something like that.