Noosphere89

Karma: 501

Noosphere89 31 Jan 2023 17:50 UTC
1 point
0 ∶ 0
in reply to: lc’s comment on: Peter Thiel’s speech at Oxford Debating Union on technological stagnation, Nuclear weapons, COVID, Environment, Alignment, ‘anti-anti anti-anti-classical liberalism’, Bostrom, LW, etc.
Hm, I think this is way too confident of a take here. It is possible LLMs simply can’t scale, but you need to avoid making such a rightly controversial claim as a response to someone.

Noosphere89 29 Jan 2023 19:43 UTC
1 point
0 ∶ 0
in reply to: Vitor’s comment on: Basics of Rationalist Discourse
The best frame for this is that better world models and better thinking is not free. It does require paying actual costs, of only in energy. Usually this cost can be very cheap, but things can get expensive in certain problems. Thus, costs are imposed for better reasoning by default.

Also, I think that babbling/brainstorming is pretty useless due to the high amount of dimensions for a lot of problems. Babbling and brainstorming scales as 2^n, with n being the number of dimensions, and for high input values of N, babbling and brainstorming is way too expensive. It’s similar to 2 of John Wentworth’s posts that I’ll link below, but in most real problems, babbling and brainstorming will make progress way too slowly to be of any relevance.

This is also why random exploration is so bad compared to focused exploration.

Links below for why I believe in the idea that babbling/brainstorming is usually not worth it:

https://www.lesswrong.com/posts/4XRjPocTprL4L8tmB/science-in-a-high-dimensional-world#comments

https://www.lesswrong.com/posts/pT48swb8LoPowiAzR/everyday-lessons-from-high-dimensional-optimization

Noosphere89 29 Jan 2023 1:44 UTC
1 point
in reply to: Alex Flint’s comment on: Soares, Tallinn, and Yudkowsky discuss AGI cognition
RE the FOOM debate: On this, I think the Hansonian viewpoint that takeoff would be gradual was way more correct than the discontinuous narrative of Eliezer, where AI progress in the real world follows more of a Hansonian path.

Eliezer didn’t get this totally wrong, and there are some results in AI showing that there can be phase transitions/discontinuities. But overall, a good prior for AI progress is that it will look like the Hansonian continuous progress rather than the FOOM of Eliezer.

Noosphere89 27 Jan 2023 20:13 UTC
1 point
0 ∶ 0
in reply to: Raemon’s comment on: Ukraine/Russia Conflict (2022)
While I agree that political stuff should mostly be avoided, I also believe that we are here to model reality. And thus calling it a war accurately describes what is going on, independently of who you want to win, so I do think this is a fairly safe change.

Yes, politics should mostly be avoided. But this is more in the domain of moral/opinionated politics, rather than what is actually going on. Accuracy is a strength here, and so long as we avoid talking about what political/moral outcomes you want, then I do think it’s fairly safe to edit political areas.

Noosphere89 27 Jan 2023 17:57 UTC
1 point
0 ∶ 0
in reply to: paulfchristiano’s comment on: Thoughts on the impact of RLHF research
Sorry to respond late, but a crux I might have here is that I see the removal of myopia and the addition of agency/non-causal decision theories as a major negative of an alignment plan by default, and without specific mechanisms of how deceptive alignment/mesa optimizers can’t arise, I expect non-myopic training to find such things.

In general, the fact that OpenAI chose RLHF made the problem quite harder, and I suspect this is an example of Goodhart’s law in action.

The Recursive Reward Modeling and debate plans could make up for this, assuming we can solve deceptive alignment. But right now, I see trouble ahead and OpenAI is probably going to be bailed out by other alignment groups.

Noosphere89 27 Jan 2023 17:05 UTC
1 point
0 ∶ 0
in reply to: Edward P. Könings’s comment on: Thatcher’s Axiom

An alignment must be done through coordination between individuals. However, this seems to me to be an extremely difficult thing to do.

Personally, I’d eschew that and instead moderate my goals/reframe the goals of alignment.

Noosphere89 27 Jan 2023 14:19 UTC
1 point
0 ∶ 0
on: Thatcher’s Axiom
One important implication of this post relating to AI Alignment: It is impossible for AI to be aligned with society, conditional on the individuals not being all aligned with each other. Only in the N=1 case can guaranteed alignment be achieved.

In the pointers ontology, you can’t point to a real world thing that is a society, culture or group having preferences or values, unless all members have the same preferences.

And thus we need to be more modest in our alignment ambitions. Only AI aligned to individuals is at all feasibly possible. And that makes the technical alignment groups look way better.

It’s also the best retort to attempted collectivist cultures and societies.

Noosphere89 27 Jan 2023 1:50 UTC
1 point
0 ∶ 0
on: The recent banality of rationality (and effective altruism)
IMO, the series of crises in the EA movement is at partly related to the fact that the EA movement is obviously having issues.

With that stated, I think EA probably can’t do well for a very basic reason: The idea that you can improve or harm society’s preferences, because there are no preferences of society, since it doesn’t exist. Thus I think EA is trying to do an impossible goal.

This is explored in the post below on Thatcher’s Axiom:

https://www.lesswrong.com/posts/YYuB8w4nrfWmLzNob/thatcher-s-axiom

IMO, I think the rationality project/LW is handling these crises far better than EA is doing.

Noosphere89 26 Jan 2023 18:38 UTC
1 point
0 ∶ 0
in reply to: paulfchristiano’s comment on: Thoughts on the impact of RLHF research
Basically, I’m talking about how RLHF removed a very valuable property called myopia. If you had myopia by default, like say the GPT series of simulators, then you just had to apply the appropriate decision theory like LCDT, and the GPT series of simulators could do something like HCH or IDA on real life. But RLHF removed myopia, and thus deceptive alignment and mesa optimization is possible, arguably incentivized under a non-myopic scheme. This is probably harder to solve than having a non-agentic system alignment problem.

I’ll provide a link below:

https://www.lesswrong.com/posts/yRAo2KEGWenKYZG9K/discovering-language-model-behaviors-with-model-written

Now you do mention that RLHF is more capable, and yeah that is sort of depressing that the most capable models align well with the most deceptive models.

Noosphere89 25 Jan 2023 23:00 UTC
1 point
0 ∶ 0
in reply to: Garrett Baker’s comment on: Thoughts on the impact of RLHF research
And this is actually pretty reasonable as a strategy, given their general myopia by default and their simulator nature playing well with alignment ideas like HCH. If we could avoid a second optimizer arising, then this scaled up would be nearly ideal for automated research on say alignment. But RLHF ruined it, and this was IMO a good example of a looking good alignment strategy that wasn’t actually good.

Noosphere89 25 Jan 2023 13:48 UTC
2 points
0 ∶ 0
in reply to: JBlack’s comment on: Thatcher’s Axiom
This is only true if all agents share the same goal/are aligned. If not, then there is no way to set a social utility curve such that improving or declining utility, that is preferences are possible.

Noosphere89 24 Jan 2023 23:13 UTC
8 points
0 ∶ 0
on: Thatcher’s Axiom
The summary of this post: Concepts like improving or harming society or abstract groups is a type error, and one should instead think of redistribution.

If the agents are aligned, than optimal redistribution is possible.

If not, then this reduces to a political and redistributive conflict.

But the idea that a society exists that can have preferences made worse or better is a type error.

Noosphere89 24 Jan 2023 19:49 UTC
1 point
0 ∶ 0
in reply to: Jacob Pfau’s comment on: What a compute-centric framework says about AI takeoff speeds—draft report
Yeah, I agree that the disagreement is probably more important to resolve, and I haven’t much addressed that.

Noosphere89 24 Jan 2023 18:53 UTC
3 points
0 ∶ 0
in reply to: Jacob Pfau’s comment on: What a compute-centric framework says about AI takeoff speeds—draft report
Basically, a rational reason to have longer timelines is the fact that there’s a non-trivial chance that safety researchers are wrong due to selection effects, community epistemic problems, and overestimating the impact of AGI.

Link below:

https://forum.effectivealtruism.org/posts/L6ZmggEJw8ri4KB8X/my-highly-personal-skepticism-braindump-on-existential-risk#comments

Noosphere89 22 Jan 2023 14:10 UTC
2 points
0 ∶ 0
in reply to: Heighn’s comment on: Transcript of Sam Altman’s interview touching on AI safety
It basically comes down to the fact that agents using too smart decision theories like FDT or UDT can fundamentally be deceptively aligned, even if myopia is retained by default.

That’s the problem with one-boxing in Newcomb’s problem, because it implies that our GPTs could very well become deceptively aligned.

Link below:

https://www.lesswrong.com/posts/LCLBnmwdxkkz5fNvH/open-problems-with-myopia

The LCDT decision theory does prevent deception, assuming it’s implemented correctly.

Link below:

https://www.lesswrong.com/posts/Y76durQHrfqwgwM5o/lcdt-a-myopic-decision-theory

Noosphere89 21 Jan 2023 19:02 UTC
−1 points
in reply to: abramdemski’s comment on: A voting theory primer for rationalists
Basically, a utility monster is a person or group that derives orders of magnitude more utility from some activity, and the effect is so large that it cancels out the rest of the population’s preferences.

An example is Confederate slave owners derived way more utility from owning slaves than non-slave owners, so if there was an election over whether slavery is illegal, then the Confederate slave owners have a strategy that is: Turn into utility monsters that derive orders of magnitude more utility, and make slave owners have extreme preferences. Even if there were 1,000 slave owners to 1 million non slave owners, there’s still a way for the slave owners to win using a strategy.

Noosphere89 21 Jan 2023 18:17 UTC
1 point
0 ∶ 0
in reply to: Dan H’s comment on: Transcript of Sam Altman’s interview touching on AI safety
It’s from the post: Discovering Language Model Behaviors with Model-Written Evaluations, where they have this to say about it:

non-CDT-style reasoning (e.g. one-boxing on Newcomb’s problem).

Basically, the AI is intending to one-box on Newcomb’s problem, which is a sure sign of non-causal decision theories, since causal decision theory chooses to two-box on Newcomb’s problem.

Link below:

https://www.lesswrong.com/posts/yRAo2KEGWenKYZG9K/discovering-language-model-behaviors-with-model-written

Noosphere89 21 Jan 2023 16:33 UTC
0 points
0 ∶ 0
in reply to: Dan H’s comment on: Transcript of Sam Altman’s interview touching on AI safety
Much more importantly, I think RLHF has backfired in general, due to breaking myopia and making them have non-causal decision theories, and only condition is necessary to make this alignment scheme net negative.

Noosphere89 20 Jan 2023 18:37 UTC
3 points
0 ∶ 0
in reply to: Akash’s comment on: Transcript of Sam Altman’s interview touching on AI safety

But I think it’s more subtle than most people think. You hear a lot of people talk about AI capabilities and AI alignment as in orthogonal vectors. You’re bad if you’re a capabilities researcher and you’re good if you’re an alignment researcher. It actually sounds very reasonable, but they’re almost the same thing. Deep learning is just gonna solve all of these problems and so far that’s what the progress has been. And progress on capabilities is also what has let us make the systems safer and vice versa surprisingly. So I think none of the sort of sound-bite easy answers work.

Pointing this out, since I don’t nearly agree with this, IMO, at least not strongly enough that capabilities and safety are the same thing. Also, I note a motivated reasoning alert here, since this is what someone would write to make sure that their beliefs around AI capabilities are good is reinforced, since the inconvenient world where the Orthogonality Thesis and instrumental convergence is true would be personally disastrous for OpenAI.

Noosphere89 20 Jan 2023 15:53 UTC
4 points
0 ∶ 0
on: Critique of some recent philosophy of LLMs’ minds

Reductionism. “LLMs are just mathematical models (matrix multiplication, calculations on GPU, next token prediction, Chinese room, stochastic parrot), which doesn’t understand, doesn’t think, have no meaning, etc.” Unfortunately, this is the level at which a lot of A(G)I researchers are.

What people often forget is that at the very lowest levels, all intelligent, computational systems approximating a Universal Turing Machine are built by unintelligent gates and matrix multiplication. Indeed, Embedded Agency shows why a very unintelligent system can be made into intelligent and conscious solely via use of internal memory and parallelization.