The opening sounds a lot like saying “aerodynamics used to be a science until people started building planes.”
The idea that an area of study is less scientific because the subject is inelegant is a blinkered view of what science is. A physicist’s view. It is one I’m deeply sympathetic to, and if your definition of science is Rutherford’s, you might be right, but a reasonable one that includes chemistry would have to include AI as well.
Related: I read that lots of cultures seem to independently consider there to have originally been seven Pleiades though one can only count six with the unaided eye, and have myths about what happened to the seventh. (link) Two of the bright ones are too close to resolve, and the idea is that the story dates back to a time when they were farther apart.
Calling it “eldritch” is mere rhetorical flourish to evoke Lovecraft; of course it’s not literally paranormal.
Asking which individual is responsible for the evil of the state is like asking which transistor in the AGI is misaligned. That kind of reductionism obviously won’t get you a reasonable answer, and you know it.
The problem is the incorrigibility of the system; it’s the same idea as Scott Alexander’s Meditations on Moloch: ultimately, it’s a coordination problem that giving a name to helps reify in the human mind. In this context, I like quoting 1 Samuel 8:11–18, slightly paraphrased.
Upon consideration, a lighthearted example might be better than Cabaret at showing how such a system can get out of control: Daylight Saving Time. The US government can’t get rid of it, and even I can’t find a way to blame that on bureaucrats.
As for “the state, it is us,” I guess I don’t understand what you mean. Sure, in the same sense the serfs are the estate; the conscripts, the army; and the slaves, the plantation, but that’s not really what one means when denouncing the state. But if it’s a way of transferring moral culpability for the state’s actions, I reject that entirely: I do not blame a slave for his submission.
I don’t think the moral intuitions of the people involved are really relevant, for the reasons above, but more importantly, you say that looking backwards through human history, the distribution is not fundamentally unpredictable: that’s not how predictions work! To people who had never seen a state before, I claim it would have indeed have been alien to see, for example, fishermen killing whales for neither meat nor blubber, but to fill quotas.
If AGI is indeed as “fundamentally different” as you worry from anything that came before it, I do not consider it as unlikely to be better than the state as it exists today as you do.
Scenario C seemed the most promising to me too, but for different reasons.
For even more brevity with no loss of substance:
A turkey gets fed every day, right up until it’s slaughtered before Thanksgiving.
even though I have infinite energy I’m not giving you a personal high resolution paradise simulation
If your definition of the term “love” encompasses this kind of pointless cruelty, I question the coherence of concept you think it describes.
You’re thinking too high-tech. This kind of reasoning would logically lead to opposing college degrees in the sciences.
It’s strikes me as a little … unthinking to single the kid out for “abusing” a fish when he joined a group of other people fishing.
(Also, why does the fish go back? Is the point not to eat the fish you catch? If it were me, and you’re eating the ones the other kids catch, but throwing mine back, I’d keep catching it over and over too, simply out of principle/spite.)
apologizing doesn’t automatically that you deserve forgiveness.
I like the perspective of the Andalite forgiveness ritual, from Animorphs, which goes something like, “I have fixed all that can be fixed, I have learned all that can be learned, I have sworn not to repeat my mistake, and I now claim the forgiveness that is mine by right.”
Under this view, it is not up to those you have wronged to choose to forgive you; at best, they can decide only whether or not to acknowledge that you have been forgiven (through doing the things in the ritual). You might still care about that, of course, but that’s now a separate thing.
I like the Voltaire (actually Evelyn Beatrice Hall) quote as much as the next guy, but if you would punish me for honestly saying what I believe (“freedom of speech does not mean freedom from consequences”), “Fuck you, I’m gonna lie to you” is the morally sound response. That works even if it’s society as a whole punishing me: I reject entirely your supposed duty to honestly state one’s beliefs.
“hey, your kids have a real chance of dying in the next decade”
Yes, every four years, if the good guys don’t win the next (US) presidential election. Or if people don’t switch to/away from nuclear power. Or they’re killed by immigrants/cops. Or they die of a fentanyl overdose. Or in a school shooting. Or if the Iraqis/Russians/Chinese invade. Or if taxes are lowered/raised.
Perhaps telling people they or their children are going to die imminently isn’t a standard tactic of “mere politics” where you are; you did say you’re not American.
Or if they do not choose what you want them to choose? If they want something different than what you want
“Aligned to whom?” remains the question in the field of “AI alignment,” and as it has been since the beginning it continues to be hand-waved away.
Okay, so this is another instance of plenty of high-status people with a lot to lose declining to say they believe something that would get them fired and/or ostracized. Do you believe it has any bearing on whether or not it is true?
The “… I accidentally clicked a radio button and can’t un-answer this question” option is funny: it’s a good solution to a common problem, but one that doesn’t actually exist in this case.
Lebanon tried this “balancing different factions” thing. Their system probably won’t survive a census: the last one was in 1932.
This is a good (and pretty standard) policy for any secret shared with you, of which “exfohazard” is, if not synonymous, a subcategory.
On the topic of improving the voting mechanism, I propose that strong votes, up or down, be public, like reactions are.
Is there a prediction market on whether he’ll be reinstated?
“You wouldn’t make that kind of mistake about an AI” is a good line. Almost no one else thinks of the state as an eldritch horror, or even refers to it as such in jest, like they do about AI systems. They should.