I’m not sure what’s the best solution in general. For this post specifically, maybe we could drop the narration?
cousin_it(Vladimir Slepnev)
Karma: 29,230
I’m not sure international coordination is the right place to start. If the Chinese are working on a technology that will end humanity, that doesn’t mean the US needs to work on the same technology. There’s no point working on such technology. The US could just stop. That would send an important signal: “We believe this technology is so dangerous that nobody should develop it, so we’re stopping work on it, and asking everyone else to stop as well.” After that, the next step could be: “We believe that anyone else working on this technology is endangering humanity as well, so we’d like to negotiate with them on stopping, and we’re prepared to act with force if negotiations fail.”
Yeah, I think the narration doesn’t catch up when I edit the post, and I’ve edited it a lot. Maybe there’s a button to refresh it but I haven’t found it. @habryka?
I think hoping for “pseudokindness” doesn’t really work. You can have one-millionth care about a flower, but you’ll still pave it over if you have more-than-one-millionth desire for a parking lot there. And if we’re counting on AIs to have certain drives in tiny amounts, we shouldn’t just talk about kindness, but also for example desire for justice (leading to punishment and s-risk). So putting our hopes on these one-millionths feels really risky.
(Edited because my previous reply was a bit off the mark.)
I don’t think this scenario depends on government. If AI is better at all jobs and can make more efficient use of all resources, “AI does all jobs and uses all resources” is the efficient market outcome. All that’s needed is that companies align their AIs to the company’s money interest, and people use and adapt AI in the pursuit of money interest. Which is what’s happening now.
A single AI taking dramatic transformative action seems less likely to me, because it’ll have to take place in a world already planted thick with AI and near-AI following money interests.
I’m not sure what exactly you’re proposing to transfer to the state.
-
The right to charge rent on the land? But under Georgism the state already owns that right.
-
The structure you built on the land? But that structure, without the land, is a depreciating asset. Chances are the next user of the land will just tear it down and build something else. So you might not get enough money for retirement by offering up the structure alone.
-
I think that response basically doesn’t work. But when I started writing in more detail why it doesn’t work, it morphed into a book review that I’ve wanted to write for the last couple years but always put it off. So thank you for finally making me write it!
Book review: The Quincunx
Page 87:
The clusters can be built in the US, and we have to get our act together to make sure it happens in the US.
No, we have to make sure it doesn’t happen anywhere.
Page 110:
What we want is to add side-constraints: don’t lie, don’t break the law, etc.
That’s very not enough. A superintelligence will be much more economically powerful than humans. If it merely exhibits normal human levels of benevolence, truth-telling, law-obeying, money-seeking, power-seeking and so on, it will deprive humans of everything.
It’s entirely legal to do jobs so cheaply that others can’t compete, and to show people optimized messages to make them spend savings on consumption. A superintelligence merely doing these two things superhumanly well, staying within the law, is sufficient to deprive most people of everything. Moreover, the money incentives point to building superintelligences that will do exactly these things, while rushing to the market and spending the minimum on alignment.
Superintelligence requires super-benevolence. It must not be built for profit or for an arms race, it has to be built for good as the central goal. We’ve been saying this for decades. If AI researchers even now keep talking in terms like “add constraints to not break the law”, we really are fucked.
Simulations; predictors (not necessarily perfect); amnesia; identical copies; players with aligned interests.
Why isn’t incremental progress at instilling human-like behavior into machines, incremental progress on AGI alignment?
It kind of is, but unfortunately treating others badly when you have lots of power is also part of human nature. And there’s no real limit to how bad it could get, see the Belgian Congo for example.
An elected representative who’s term limit is coming up wouldn’t have the same incentives.
I think this proves too much. If elected representatives only follow self-interest, then democracy is pointless to begin with, because any representative once elected will simply obey the highest bidder. Democracy works to the extent that people vote for representatives who represent the people’s interests, which do reach beyond the term limit.
Don’t know about others, but to me feels like “wokeness” has faded from view a bit because economic inequality has become the main issue again. And I agree with that: making people less dependent on landlords and employers is indeed the main issue. (The AI issue is more urgent, but even the AI transition would feel safer to me if we had a system where people’s livelihoods weren’t tied to jobs or affording rent.)
If orchestral jobs for violinists are so scarce, what do you think about the options of branching out? Playing with a band, going into pop violin, etc.
I think Scott’s original story described scissor statements a bit differently. The people reading them thought “hmm, this isn’t controversial at all, this is just obviously true, maybe the scissor-statement-generator has a bug”. And then other people read the same statement and said it was obviously false, and controversy resulted. Like the black and blue vs white and gold dress, or yanny/laurel. Maybe today’s LLMs aren’t yet smart enough to come up with new such statements.
EDIT: I think one possible reason why LLMs have trouble with this kind of question (and really with any question that requires coming up with specific interesting things) is that they have a bias toward generic. In my interactions with them at least, I keep having to constrain the question with specificity, and then the model will still try to give the most generic answer it can get away with.
Yeah. To me your post first read like it was making a historical claim—about gradual voluntary self-disarmament. But maybe I misread and you only intended to make the smaller point about “getting epsilon more from participation”, in that case yeah, my criticism is off target and sorry.
”this system was designed such that every participant was getting an epsilon more from participation than they expected from breaking its rules”
I do agree with this as stated. But a system can be very coercive and still meet the letter of this (by making “what is expected from breaking the rules” really bad). So maybe you need a stronger statement.
You don’t need to weaken your views, they can be criticized just fine as they are. My main criticism is that you believe in a kind of social contract that includes and benefits most people, but I think in reality there’s much more coercion, much more rules that benefit the powerful at the expense of everyone else. For example, the whole system of land ownership and rent would look very different if it was designed with majority interests in mind.
I like reviews of imaginary books as much as the next guy, but I’m a bit miffed that you didn’t do it the cool way: by recounting a point from the book and then saying “the author is wrong and a moron, actually things are this way”, then doing the same for the next point and so on. This way the review wouldn’t come off as being fawning toward your own ideas (which let’s face it is a bit weird), and also the readers would get a valuable rationality exercise in figuring out who’s the moron in each instance. Bonus points if you yourself genuinely don’t know who’s the moron—that can elevate the whole thing into art.
It still seems to me that international cooperation isn’t the right first step. If the US believes that AI is potentially world-ending, it should put its money where its mouth is, and first set up a national commission with the power to check AIs and AI training runs for safety, and ban them if needed. Then China will plausibly do the same as well, and from a cooperation of like-minded people in both countries’ safety commissions we can maybe get an international commission. But if you skip this first step, then China’s negotiators can reasonably say: why do you ask us for cooperation while you still continue AI development unchecked? This shows you don’t really believe it’s dangerous, and are just trying to gain an advantage.