Chris Leong

Karma: 226

Chris Leong 15 Mar 2023 0:00 UTC
1 point
0
on: GPT can write Quines now (GPT-4)
I would love to see more experimentation here to determine whether GPT4 can do more complicated quines that are less likely to be able to be copied. For example, we could insist that it includes a certain string or avoids certain functions.

Chris Leong 1 Mar 2023 17:32 UTC
1 point
0
in reply to: Vanessa Kosoy’s comment on: A mostly critical review of infra-Bayesianism
Thanks for the detailed response.

To be honest, I’ve been persuaded that we disagree enough in our fundamental philosophical approaches, that I’m not planning to deeply dive into infrabayesianism, so I can’t respond to many of your technical points (though I am planning to read the remaining parts of Thomas Larson’s summary and see if any of your talks have been recorded).

“However, CDT and EDT are both too toyish for this purpose, since they ignore learning and instead assume the agent already knows how the world works, and moreover this knowledge is represented in the preferable form of the corresponding decision theory”—this is one insight I took from infrabayesianism. I would have highlighted this in my comment, but I forgot to mention it.

“ Learning requires that an agent that sees an observation which is impossible according to hypothesis H, discards hypothesis H and acts on the other hypotheses in has”—I have higher expectations from learning agents—that they learn to solve such problems despite the difficulties.

Chris Leong 1 Mar 2023 8:05 UTC
1 point
0
on: A mostly critical review of infra-Bayesianism
I’m curious why you say it handles Newcomb’s problem well. The Nirvana trick seems like an artificial intervention where we manually assign certain situations a utility of infinity to enforce a consistent condition which then ensures they are ignored when calculating the maximin. If we are manually intervening, why not just manually cross out the cases we wish to ignore, instead of adding them with infinite value then immediately ignoring them.

Just because we modelled this using infrabayesianism, it doesn’t follow that it contributed anything to the solution. It feels like we just got out what we put in, but that this is obscured by a philosophical shell game. The reason why it feels compelling is though we’re only adding in an option to then immediately ignore it, this is sufficient to give us a fake sense of having made a non-trivial decision.

It would seem that infrabayesianism might be contributing to our understanding of the problem if the infinite utility arose organically, but as far as I can tell, this is a purely artificial intervention.

I think this is made clearer by Thomas Larson’s explanation of infrabayesianism failing Transparent Newcomb’s. It seems clear to me that this isn’t an edge case; instead it demonstrates that rather than solving counterfactuals, all this trick does is give you back what you put in (one-boxing in the case where you see proof you one-box, two-boxing in the case where you see proof you two-box).

(Vanessa claims to have a new intervention that makes the Nirvana trick redundant, if this doesn’t fall prey to the same issues, I’d love to know)

Chris Leong 21 Feb 2023 0:05 UTC
4 points
3
in reply to: TinkerBird’s comment on: Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky
I’d encourage you to do that.

Chris Leong 27 Jan 2023 3:34 UTC
18 points
11
on: Alexander and Yudkowsky on AGI goals
I wish Eliezer had been clearer on why we can’t produce an AI that internalises human morality with gradient descent. I agree gradient descent is not the same as a combination of evolutionary learning + within lifetime learning, but it wasn’t clear to me why this meant that no combination of training schedule and/or bias could produce something similar.

Chris Leong 21 Jan 2023 0:26 UTC
2 points
0
on: Shard theory alignment requires magic.
Why do you believe that “But if you push the complexity up too fast, the RL process will fail, or the AI will be more likely to learn heuristics that are better than nothing but aren’t what we intended”?

I understand why this could cause the AI to fail, but why might it learn incorrect heuristics?

Chris Leong 31 Dec 2022 12:17 UTC
1 point
0
in reply to: shminux’s comment on: Two Issues with Playing Chicken with the Universe
I’m discussing an agent that does in fact take 5 which imagines taking 10 instead. There have been some discussions of decision theory using proof-based agents and how they can run in spurious counterfactual. If you’re confused, you can try searching the archive of this website. I tried earlier today, but couldn’t find particularly good resources to recommend. I couldn’t find a good resource for playing chicken with the universe either.

(I may write a proper article at some point in the future to explain these concepts if I can’t find an article that explains them well)

Chris Leong 29 Dec 2022 5:27 UTC
1 point
0
on: Shard Theory in Nine Theses: a Distillation and Critical Appraisal
I’ve been reading a lot of shared theory and finding fascinating, even though I’m not convinced that a more agentic subcomponent wouldn’t win out and make the shards obsolete. Especially if you’re claiming the more agentic shards would seize power, then surely an actual agent would as well?

I would love to hear why Team Shard believes that the shards will survive in any significant form. Off the top of my head, I’d guess their explanation might relate to how shards haven’t disappeared in humans?

On the other hand, I much more strongly endorse the notion of thinking of a reward as a chisel rather than something we’ll likely find a way of optimising.

Chris Leong 29 Dec 2022 5:22 UTC
7 points
0
in reply to: So8res’s comment on: Shard Theory in Nine Theses: a Distillation and Critical Appraisal
Are there any agendas you would particularly like to see distilled?

Chris Leong 27 Dec 2022 5:46 UTC
4 points
on: Mazes and Duality [Transcript]
What does it mean for a constraint to be low-dimensional?

Chris Leong 26 Dec 2022 1:23 UTC
2 points
3
in reply to: Noosphere89’s comment on: I’ve updated towards AI boxing being surprisingly easy
Filtering out all the implicit information would be really, really hard.

Chris Leong 26 Dec 2022 1:03 UTC
1 point
0
on: I’ve updated towards AI boxing being surprisingly easy
I think the key problem is that this sandboxing won’t work for anything with a large language model as a component.

Chris Leong 19 Dec 2022 23:54 UTC
6 points
4
on: Towards Hodge-podge Alignment
Very happy to see someone writing about this as I’ve been thinking that there should be more research into this for a while. I guess my main doubt with this strategy is that if a system is running for long enough in a wide enough variety of circumstances maybe certain rare outcomes are virtually guaranteed?

Chris Leong 11 Dec 2022 5:47 UTC
2 points
0
on: Take 8: Queer the inner/outer alignment dichotomy.
I’d encourage you to delve more into this paragraph as I think this is the part of your article where it becomes the most hand-wavey:

“In order to “really solve” outer alignment, you want the AI-optimization process to care about the generalization properties of the created AI beyond the training data. In order to “really solve” inner alignment, the created AI shouldn’t just care about the raw outputs of the process that created it, it should care about the things communicated by the AI-optimization process in its real-world context.”

[Question] What are the major underlying divisions in AI safety?

Chris Leong6 Dec 2022 3:28 UTC

5 points

2 comments1 min readLW link

Chris Leong 6 Oct 2022 23:02 UTC
3 points
6
on: So, geez there’s a lot of AI content these days
I don’t know, but sounds like an obvious use case for a sub forum? The solutions listed above seem hackish.

Chris Leong 20 Sep 2022 12:46 UTC
3 points
0
on: Alignment via prosocial brain algorithms
I have to be honest, I’m skeptical. If we study how human prosociality works, my expectation is that we learn enough to produce some toy models with some very simplistic pro-sociality, but this seems insufficient for generating an AI capable of navigating tough moral dilemmas; just situations sufficiently off-distribution. The reason why we want humans in the loop is not because they are vaguely pro-social but because of the ability of humans to handle novel situations.

———————————————-

Actually, I shouldn’t completely rule out the value of this research. I think that it’s failure will be an utterly boring result, but perhaps there is value in seeing how it fails concretely. That said, it’s unlikely to be worth the effort if it takes a lot of neuroscience research just to construct these examples of failure.

Chris Leong 27 Aug 2022 3:19 UTC
1 point
0
on: Matt Yglesias on AI Policy
I’m sure if he spent five minutes brainstorming he could come up with more things, or maybe I’m just wrongly calibrated on how much agency people have?

Chris Leong 26 Jun 2022 5:59 UTC
1 point
in reply to: bryjnar’s comment on: Half-baked AI Safety ideas thread
I’ve had similar thoughts too. I guess the way I’d implement it is by giving the AI a command that it can activate that directly overwrites the reward buffer but then turns the AI off. The idea here is to make it as easy as possible for an ai inclined to wire head to actually wire head so it is less incentivised to act in the physical world.

During training I would ensure that the SGD used the true reward rather than the wire-headed reward. Maybe that would be sufficient to stop wire-heading, but there are issues with it pursuing the highest probability plan rather than just a high probability plan. Maybe quantilising probability can help here

Chris Leong 24 Apr 2022 9:51 UTC
6 points
in reply to: Dagon’s comment on: The New Right appears to be on the rise for better or worse
Theres a difference between debating the merits of different political positions and merely announcing an apparent trend. I’m doing the later and I don’t think the risks associated with this are too severe. So it’s not exactly open season.

Chris Leong

[Question] What are the ma­jor un­der­ly­ing di­vi­sions in AI safety?

[Question] What are the major underlying divisions in AI safety?