Is plushcare a thing for people in Europe? Can’t find an answer via googling.
Rafael Harth
Karma: 3,134
What information is required to determine a system’s consciousness (or lack thereof)? Consider voting in this poll with what you think.
(You can view the results without voting.)
I mean, this isn’t a big deal either way, but I suspect that the exposure of the median LWer to game theory consists primarily of the prisoner’s dilemma, where “cooperate” is the standard name for one of the two actions.
Regardless, the original post is correct in it’s use. “cooperation” can be among a subgroup, even if it’s antagonistic or lower-value to other subgroups or the naive sum of the whole.
I also wondered something like this when reading, but this felt like a bad reason to use the term. But, well, it really just comes down to what you associate with the term.
Isn’t this more about coordination rather than cooperation? In game theory, cooperation is specifically about doing a nice thing for someone else.
Not at all. Having fixed points proves LFW wrong, not right.
Okay; I got the impression that you had it backward from the post, one reason was this quote:
Fortunately for the argument, the assumption of continuity for g is plausible in some real-world settings
Where I thought “the argument” refers to Caplan, but continuity is bad news for Caplan, not good news.
Anyway, if your point is that showing the existence of fixed points would disprove LFW (the contrapositive of LFW → no fixed points], then I take back what I said about your post not being relevant for LFW. However, I maintain that Caplan’s argument has no merit either way.
Right, I was just thinking negation, but yeah the formalization doesn’t make a lot of sense. (Also, there was some frustration with Caplan bleeding into my comment, apologies.)
If you don’t make the physics simulator causally entangled with Caplan then his argument doesn’t work at all.
Yes, because it doesn’t work at all!
Bryan is saying he can avoid fixed points. This is trivially true in the deterministic setting, and your post is asking whether it’s also true in the probabilistic setting. But it doesn’t matter whether it’s true in the probabilistic setting! The entanglement argument shows that even if you have a function without fixed points, this still doesn’t give you LFW.
(I think you got confused in the post and thought that having fixed points shows LFW, but if anything it’s the opposite; Bryan’s point is that he can avoid fixed points. So the deterministic setting is the more favorable one for his argument, and since it doesn’t work there, it just doesn’t work period.)
I think this argument doesn’t deserve anywhere near as much thought as you’ve given it. Caplan is committing a logical error, nothing else.
He probably reasoned as follows:
-
If determinism is true, I am computable.
-
Therefore, a large enough computer can compute what I will say.
-
Since my reaction is just more physics, those should be computable as well, hence it should also be possible to tell me what I will do after hearing the result.
This is wrong because “what Caplan outputs after seeing the prediction of our physics simualtor” is a system larger than the physics simulator and hence not computable by the physics simulator. Caplan’s thought experiment works as soon as you make it so the physics simulator is not causally entangled with Caplan.
I don’t think fixed points have any place in this analysis. Obviously, Caplan can choose to implement a function without a fixed point, like (edit: rather ), in fact he’s saying this in the comment you quoted. The question is why he can do this, since (as by the above) he supposedly can’t.
See also my phrasing of this problem and Richard_Kennaway’s answer. I think the real problem with his quote is that it’s so badly phrased that the argument isn’t even explicit, which paradoxically makes it harder to refute. You first have to reconstruct the argument, and then it gets easier to see why it’s wrong. But I don’t think there’s anything interesting there.
-
I think I’m missing the most important part of this debate. How does the second hose help? The air outside is hot; with one hose, hot air enters the house because of the vacuum effect; with two hoses, the second hose explicitly sucks in air from the outside… which is still hot. Where is the difference?
Using it to do neural annealing was definitely the biggest boost.
Other than that, the step where I shifted the focus from “let’s try to do well in formal practice” to “let’s focus primarily on integrating it into life (mostly focused on equanimity).
I think control is a meaningful concept. You could have AI that doesn’t try to alter your terminal goals. Something that just does what you want (not what you ask, since that has well-known failure modes) without trying to persuade you into something else.
The difficulty of building such a system is another question, alas.
I think I’ve experienced massive benefits, both in reduced suffering an increased productivity.
Makes sense. I’ve always been using “G” more literally as “general”, but it’s certainly possible that this is unusual.
How is GPT-3 not general? It seems like the I should be more in doubt than the G.
Yeah, if I got to decide, I would barely factor in how bad the world is right now. Delay AGI until it’s outweighed by other x-risks.
Yes, but I didn’t mean to ask whether it’s relevant, I meant to ask whether it’s accurate. Does the output of language models, in fact, feel like this? Seemed like something relevant to ask you since you’ve seen lots of text completions.
And if it does, what is the reason for not having long timelines? If neural networks only solved the easy part of the problem, that implies that they’re a much smaller step toward AGI than many argued recently.
Do you think this describes language models?
I think what you get is a person talking with no inhibitions whatsoever. Language models don’t match that.
What do you picture a language model with no inhibitions to look like? Because if I try to imagine it, then “something that outputs reasonable sounding text until sooner or later it fails hard” seems to be a decent fit. Of course haven’t thought much about the generator/assessor distinction.
I mean, surely “inhibitions” of the language model don’t map onto human inhibitions, right? Like, a language model without the assessor module (or a much worse assessor module) is just as likely to be imitate someone who sounds unrealistically careful as someone who has no restraints.
I find your last paragraph convincing, but that of course makes me put more credence into the theory rather than less.
The orthogonality thesis says that an AI can have any combination of intelligence and goals, not that for all and . It depends entirely on how the AI is built. People like Rohin Shah assign significant probability on alignment by default, at least last I heard.
Interesting. Grammarly always points at the “This” sentences as problems in my drafts. For a while I looked at them; in approximately 100% of cases I’ve decided that the sentence is clear and made no change. Nowadays, I dismiss all such warnings without looking. This :) has sped up the process. I guess I’ll give them more consideration from now on.
Grammarly also points to some of the other things you mention in the post, which I tend to take seriously.
I suggest messaging the moderators via Intercom. They will probably change your username if you don’t do it too often.