“Case in point. He is falling behind, and he knows it. If he counts loosing SuperPower status as a complete loss condition, he has to do something drastic, and the sooner the better.”
That’s exactly one of my suspected ulterior motives: all-in desperation move. And it is an ulterior motive, since no Russian is saying “we’re taking drastic measures to assure that we don’t lose super power status in the world stage”. Quite the opposite, they’re saying “we’re ultra rational peace makers”.
On nuclear disarmament, that doesn’t hold up. If we believe that it is not a solvable problem, we shouldn’t actively work for a solution (there are plenty of think thanks theoretically working for a solution). If you begin actively working for a solution, much more tragic things could happen, like US and Russia both reach the number of nukes of 100 each, and then one of either has another 1000 in secret, or shady 3rd world leader build a nuclear arsenal of 500 and bombs the crap out of the US/Russia because being the shady 3rd world leader he is he thinks it’s worth it.
I also believe that a world without nukes would plunge right back into the perma-war craphole it was before their existence. This is not to say that nukes are good, it’s just that their absence might not be a lot better.
″ I think that the vast majority of a person’s impact in this world is what hyperbeings he chooses to host/align to, with object level reality, barely even mattering.”
I agree that personal values (no need to mystify) are important, but action is equally important. You can be very virtuous, but if you don’t take action (by, for instance, falling for the Buddhist-like fallacy that sitting down and meditating will eventually save the world by itself), your impact will be minor. Specially in critical times like this. Maybe sitting down and meditating would be ok centuries ago where no transformative technologies were in sight. Now, with transformative technologies decade(s) off, it’s totally different. We do have to save the world.
“I assert that alignment is trivially easy.”
How can you control something vastly more intelligent than yourself (at least in key areas), or that can simply re-write its own code and create sub-routines, therefore bypassing your control mechanisms? Doesn’t seem easy at all. (In fact, some people like Roman Yalmpolsky have been writing papers on how it’s in fact impossible.) Even with the best compiler in the world (not no mention that nothing guarantees that progress in compilers will accompany progress in black boxes like neural networks).
“If our a measurement of “do I like this” is set to “does it kill me” I see this at best ending in a permanent boxed garden where life is basically just watched like a video and no choices matter, and nothing ever changes, and at worst becoming an eternal hell (specifically, an eternal education camp), with all of the above problems but eternal misery added on top.”
I agree with this. The alignment community is way over-focused on x-risk and way under-focused on s-risk. But after this, your position becomes a bit ambiguous. You say:
“We should work together, in public, to develop AI that we like, which will almost certainly be hostile, because only an insane and deeply confused AI would possibly be friendly to humanity. ”
What does this mean? That we should just build non-aligned AI, which would probably be hostile to humans, and therefore only kill us all instead of giving us eternal misery?
But wait, wasn’t alignment trivially easy?
Unless you mean that, since “Alignment, as I use it, isn’t becoming good. It’s becoming something recognized as good.”, then alignment, despite being easy, is a mistake and we should just build “AI that we like” which, I don’t know, being more intelligent would therefore be more virtuous than humans? But that’s extremely uncertain (as well as unlikely imo), and the stakes are way too high to take the gamble.