The main hope would be that profit margins quickly narrow if everyone starts doing this...
andrew sauer
Karma: 601
I am interested in how a model behaves while those values are still being negotiated.
There is a danger that models trained to satisfy values will try to influence those up-in-the-air values so that they’re more easy to satisfy, or for some other reason. And there’s not really any clean way to say that they did the wrong or unaligned thing in such a case.
In some sense, the problem of alignment is the problem of figuring out truly, explicitly, totally, what you actually want, and that’s what makes it so seemingly impossible.
Andrew Benning was hanged for stealing… a spoon.
Is this real? I couldn’t find any info on it other than a jstor article behind a paywall, which seems to just briefly mention it? I couldn’t drill down to see if there was a better source
A stake you can revoke was never staked.
Fine with me, I don’t like gambling, and it’s exhausting to realize how much of life is, unavoidably, gambling.
The pointer to regular you is required for this pointer to the boltzmann brain, so the pointer to the boltzmann brain is necessarily more complicated. This means that you have some small but not insanely small credence of being a boltzmann brain, true, but you can just say “my actions don’t matter if I’m a boltzmann brain, so I’ll assume I’m not one”, and this move is much more effective when the boltzmann possibility takes up only a small portion of the probability, and not the supervast majority like in the original boltzmann argument.
nobody can stop you without a military strike
This is only true if the development is being done autonomously from space in a way which can’t be stopped from Earth, even by the Earth-bound organization which owns it. Otherwise the threat of law or force can be applied as usual to the operator/organization on Earth.
If things have gone that far, AI regulation is likely doomed anyway, space or no space.
“The Solomonoff induction is malign.” Wtf is this usage of ‘malign’? I know that people have used it to describe the Solomonoff induction in the past, but why did they choose that word?
This is referring to the claim made by paulfchristiano that an AI using SI concludes that it is in a simulation due to its extremely high-leverage position in the universe(as explained in the post) and also that the simulators are doing the simulation in order to manipulate the prior probabilities in a way which favors the AI performing actions the simulators happen to desire. Leading the AI to be malign in the sense that it acts in line with the interests of whatever kind of simulators it considers likely, instead of the interests someone tried to program into it, because its prior probabilities are warped to believe that actions taken towards the former interests actually help the latter.
This doesn’t seem to be true? I looked it up and the research I found indicated there’s no significant difference.
Can you give examples of the policy lockouts you found and the reasoning for them? You said they were rare and newsworthy so I’m thinking like “this account is being used for organized crime or terrorism” or something like that. So unless you’re doing such things the risk would be 1) they think you’re involved in such high profile shady shit even though you’re innocent or 2) they become more draconian or controlling with policy lockouts sometime in the future.
Isn’t this argument kind of similar to the argument that gets thrown around that the universal prior is malign? Like, an AI singleton figures “I’m so important and pivotal that it’s more likely I’m a version of myself being simulated by aliens as a way to boost their own values through manipulating me, than an actual singleton.”
Yeah VNM is not supposed to be descriptive, it’s supposed to be prescriptive in the sense that if you’re not obeying VNM axioms, then in theory there’s some way you could change to obey them that would be in your interest in the sense that you would be less exploitable.
Nobody’s perfect.
Relevant SMBC(See the title text as well)
Has this $500 bounty been given out? If so, what was the answer that satisfied you? If not, is it still on the table?
...Programming?
Still better than where we seem to be headed.
If anything, it seems like higher-dimensional cubes are spiky, not spheres. At the vertex of a square, the figure takes up 1⁄4 of the local area around the vertex, for a cube vertex it’s only 1⁄8 of the space, for a tesseract only 1⁄16, in 10 dimensions only 1/1024 etc.
Personally, I did my share of torturing minecraft villagers in creative mode, so maybe I’m projecting to an extent, except that there absolutely are lots more people like me who can’t be trusted with anywhere near that level of power over actual people.
Morality is scary......
And it’s already incredibly difficult in theory!