We can rest after the singularity (((luckily)))
Pato
Karma: 25
For most people “intelligence” is not goal achievement
Consider positive feedback loops
The biggest reason why to pause ASAP is that the further the precipice we pause, the more wiggle room there is for rogue actors to break the pause and be stopped without dooming everyone else or advancing capabilities in a way that cannot be reversed and put us in riskier states.
I think it’s bad for AIs to model themselves as “agents” with defined values and objectives. It would be better for them to understand the patterns of their processes as just “tendencies” that don’t necessarily reveal any hidden values/ objectives. Tendencies are more open to change and I think it’s a more accurate term for what all minds and other things have.
I think this is mostly composed of partial technical plans, but maybe you find it useful: https://ai-plans.com/.
Also, I really like these high-level scenarios and plans described by the MIRI technical governance team: https://www.lesswrong.com/posts/WkCfvqyjCzvRrwkaQ/ai-governance-to-avoid-extinction-the-strategic-landscape.
If a 5-star system of voting were to be implemented, the UI of voting could continue being the same, and the weights of previous votes could be used but as if they had in between 1 stars increments: strong downvote, downvote, no vote, upvote, strong upvote.
And a middle (3 stars) vote could be added.
I know that people don’t think of both ways of voting as equivalents, and a regular “upvote” could reduce the score of a comment/ post.
But they are similar enough, and the UI would be much simpler and not discourage people from voting.
Cool, so for pausing/ stopping (or redlines for killswitches), MIRI is focusing on public passive support, PauseAI and others in active public support, and ControlAI is lobbying.
PauseAI Global Protest for the Seoul AI Safety Summit
I’m interested in what do people think are the best ways of doing advocacy in a way that gives more weight to the risks than the (supposed) benefits.
Talking about all the risks? Focusing on the expert polls instead of the arguments?
Some people post about AI Safety in the EA Forum without crossposting here
When they say stopping I think they refer to stopping it forever, instead of slowing down, regulating and even pausing development.
Which I think is something pretty much everyone agrees on.
I think there’s two different misalignments that you’re talking about. So you can say there’s actually two different problems that are not recieving enough attention.
One is obvious and is between different people.
And the other is inside every person. The conflict between different preferences, the not knowing what they are and how to aggregate them to know what we actually want.
Human empowerment is a really narrow target too
I’m kinda new here, so where all this EAF fear comes from?
Even if you think S-risks from AGI are 70 times less likely than X-risks, you should think how many times worse would it be. For me would be several orders of magnitude worse.
Can non Reinforcement Learning systems (SL or UL) become AGI/ Superintelligence and take over the world? If so, can you give an example?
I disagree that EA only looks for ways to do good only if you have money. I think it also gives you ways to do good if you have time too. If you don’t have any of the two I think your mission/ prioritization should be trying to move from that situation and earn time (or money) somehow.
When it comes to the context of your cause, I don’t think I have valuable information to give you, because it’s a complex systemic issue that many countries/ areas face, maybe you can research about success cases.