Filip Sondej

GPTs’ abil­ity to keep a se­cret is weirdly prompt-dependent

• Filip Sondej 17 Jun 2023 11:32 UTC

  3 points

  0

  in reply to: Stephen Fowler’s comment on: Scaf­folded LLMs: Less Ob­vi­ous Concerns

  1. I agree that scaffolding can take us a long way towards AGI, but I’d be very surprised if GPT4 as core model was enough.

  2. Yup, that wasn’t a critique, I just wanted to note something. By “seed of deception” I mean that the model may learn to use this ambiguity more and more, if that’s useful for passing some evals, while helping it do some computation unwanted by humans.

  3. I see, so maybe in ways which are weird to humans to think about.

• Filip Sondej 16 Jun 2023 15:11 UTC

  5 points

  0

  on: Scaf­folded LLMs: Less Ob­vi­ous Concerns

  we make the very strong assumption throughout that S-LLMs are a plausible and likely path to AGI

  It sounds unlikely and unnecessarily strong to say that we can reach AGI by scaffolding alone (if that’s what you mean). But I think it’s pretty likely that AGI will involve some amount of scaffolding, and that it will boost its capabilities significantly.

  there is a preexisting discrepancy between how humans would interpret phrases and how the base model will interpret them

  To the extent that it’s true, I expect that it may also make deception easier to arise. This discrepancy may serve as a seed of deception.

  Systems engaging in self modification will make the interpretation of their natural language data more challenging.

  Why? Sure, they will get more complex, but are there any other reasons?

  Also, I like the richness of your references in this post :)

• Filip Sondej 15 Jun 2023 18:17 UTC

  3 points

  2

  in reply to: Dagon’s comment on: Boomerang—pro­to­col to dis­solve some com­mit­ment races

  I edited the post to make it clearer that Bob throws out the wheel because he didn’t notice in time that Alice threw.

  Yup, side payments are a deviation, that’s why I have this disclaimer in game definition (I edited the post now to emphasize it more):

  there also may be some additional actions available, but they are not obvious

  Re separating speed of information and negotiations: I think here they are already pretty separate. The first example with 3 protocol rules doesn’t allow negotiations and only tackles the information speed problem. The second example with additional fourth rule enables negotiations. Maybe you could also have a system tackling only negotiations and not the information speed problem, but I’m not sure now how would it look like, or if it would be much simpler.

  Another problem (closely tied to negotiations) I wanted to tackle is something like “speed of deliberation” where agents make some bad commitments because they didn’t have enough time to consider their consequences, and later realize they want to revoke/​negotiate.

• Filip Sondej 6 Jun 2023 15:12 UTC

  3 points

  0

  in reply to: GeneSmith’s comment on: How to have Poly­geni­cally Screened Children

  Oh, so the option to choose all of those disease weights is there, it’s just a lot of effort for the parents? That’s good to know.

  Yeah, ideally it shouldn’t need to be done by each parents separately, but rather there should be existing analyses ready. And even if those orgs don’t provide a satisfactory analyses themselves, they could be done independently. F.e. collaborating on that with Happier Lives Institute could work well, as they have some similar expertise.

• Filip Sondej 5 Jun 2023 16:50 UTC

  3 points

  0

  on: How to have Poly­geni­cally Screened Children

  each disease is weighted according to its impact on disability-adjusted lifespan

  It’s a pity they don’t use some more accurate well-being metrics like f.e. WELLBY (although I think WELLBY isn’t ideal either).
  
  How much control do the parents have on what metric will be used to rank the embryos?

• Filip Sondej 4 Jun 2023 15:52 UTC

  2 points

  1

  in reply to: Dagon’s comment on: Boomerang—pro­to­col to dis­solve some com­mit­ment races

  Oh yeah, I meant the final locked-in commitment, not initial tentative one. And my point is that when committing outside is sufficiently more costly, then it’s not worth doing it, even if that would let you commit faster.

• Filip Sondej 4 Jun 2023 15:18 UTC

  1 point

  0

  in reply to: Dagon’s comment on: Boomerang—pro­to­col to dis­solve some com­mit­ment races

  Yup, you’re totally right, it may be too easy to commit in other ways, outside this protocol. But I still think it may be possible to create such a “main mechanism” for making commitments where it’s just very easy/​cheap/​credible to commit, compared to other mechanisms. But that would require a crazy amount of cooperation.

  The vast majority that I know of use ad-hoc and agent-specific commitment mechanisms

  If you have some particular mechanisms in mind could you list some? I’d like to compile a list of the most relevant commitment mechanisms to try to analyze them.

Boomerang—pro­to­col to dis­solve some com­mit­ment races

• Filip Sondej 19 May 2023 13:58 UTC

  6 points

  4

  on: Col­lec­tive Identity

  Love that post!

  Can we train ML systems that clearly manifest a collective identity?

  I feel like in multi-agent reinforcement learning that’s already the case.

  Re training setting for creating shared identity. What about a setting where a human and LLM take turns generating text, like in the current chat setting, but first they receive some task, f.e. “write a good strategy for this startup” and the context for this task. At the end they output the final answer and there is some reward model which rates the performance of the cyborg (human+LLM) as a whole.

  In practice, having real humans in this training loop may be too costly, so we may want to replace them most of the time with an imitation of a human.

  (Also a minor point to keep in mind: having emergent collective action doesn’t mean that the agents have a model of the collective self. F.e. colony of ant behaves as one, but I doubt ants have any model of the colony, rather just executing their ant procedures. Although with powerful AIs, I expect those collective self models to arise. I just mean that maybe we should be careful in transferring insight from ant colonies, swarms, hives etc., to settings with more cognitively capable agents?)

Spooky ac­tion at a dis­tance in the loss landscape

• Filip Sondej 28 Nov 2022 0:49 UTC

  1 point

  0

  in reply to: Viliam’s comment on: Guardian AI (Misal­igned sys­tems are all around us.)

  Oh yeah, definitely. I think such a system shouldn’t try to enforce one “truth”—which content is objectively good or bad.

  I’d much rather see people forming groups, each with its own moderation rules. And let people be a part of multiple groups. There’s a lot of methods that could be tried out, f.e. some groups could use algorithms like EigenTrust, to decide how much to trust users.

  But before we can get to that, I see a more prohibitive problem—that it will be hard to get enough people to get that system off the ground.

• Filip Sondej 25 Nov 2022 23:06 UTC

  3 points

  2

  on: Guardian AI (Misal­igned sys­tems are all around us.)

  Cool post! I think the minimum viable “guardian” implementation, would be to

  • embed each post/​video/​tweet into some high-dimensional space

  • find out which regions of that space are nasty (we can do this collectively—f.e. my clickbait is probably clickbaity for you too)

  • filter out those regions

  I tried to do something along these lines for youtube: https://​​github.com/​​filyp/​​yourtube

  I couldn’t find a good way to embed videos using ML, so I just scraped which videos recommend each other, and made a graph from that (which kinda is an embedding). Then I let users narrow down on some particular region of that graph. So you can not only avoid some nasty regions, but you can also decide what you want to watch right now, instead of the algorithm deciding for you. So this gives the user more autonomy.

  The accuracy isn’t yet too satisfying. I think the biggest problem with systems like these is the network effect—you could get much better results with some collaborative filtering.

• Filip Sondej 4 Nov 2022 16:05 UTC

  1 point

  0

  in reply to: Ilio’s comment on: Mind is uncountable

  Yeah, when I thought about it some more, maybe the smallest relevant physical change is a single neuron firing. Also with such a quantization, we cannot really talk about “infinitesimal” changes.

  I still think that a single neuron firing, changing the content of experience so drastically, is quite hard to swallow. There is a sense in which all that mental content should “come from” somewhere.

  I had a similar discussion with @benjamincosman, where I explore that in more detail. Here are my final thoughts from that discussion.

• Filip Sondej 4 Nov 2022 15:54 UTC

  1 point

  0

  in reply to: mako yass’s comment on: Mind is uncountable

  Oh, I’ve never stumbled on that story. Thanks for sharing it!

  I think it’s quite independent from my post (despite such a similar thought experiment) because I zoomed in on that discontinuity aspect, and Eliezer zoomed in on anthropics.

• Filip Sondej 4 Nov 2022 15:02 UTC

  1 point

  0

  in reply to: JBlack’s comment on: Mind is uncountable

  That’s a good point. I had a similar discussion with @benjamincosman, so I’ll just link my final thoughts: my comment

• Filip Sondej 4 Nov 2022 14:53 UTC

  2 points

  0

  in reply to: benjamincosman’s comment on: Mind is uncountable

  I thought about it some more, and now I think you may be right. I made an oversimplification when I implicitly assumed that a moment of experience corresponds to a physical state in some point in time. In reality, a moment of experience seems to span some duration of physical time. For example, events that happen within 100ms, are experienced as simultaneous.

  This gives some time for the physical system to implement these discontinuities (if some critical threshold was passed).

  But if this criticality happens, it should be detectable with brain imaging. So now it becomes an empirical question, that we can test.

  I still doubt the formulation in IIT, that predicts discontinious jumps in experience, regardless of whether some discontinuity physically happens or not.

  (BTW, there is a hypothetical mechanism that may implement this jump, proposed by Andres Gomez Emilsson—topological bifurcation.)

  What links here?

  • Filip Sondej's comment on Mind is uncountable by Filip Sondej (4 Nov 2022 16:05 UTC; 1 point)
  • Filip Sondej's comment on Mind is uncountable by Filip Sondej (4 Nov 2022 15:02 UTC; 1 point)

• Filip Sondej 2 Nov 2022 21:50 UTC

  1 point

  0

  in reply to: benjamincosman’s comment on: Mind is uncountable

  Hm, yeah, the smallest relevant physical difference may actually be one neuron firing, not one moved atom.

  ---

  What I meant by between them, was that there would need to be some third substrate that is neither physical nor mental, and produces this jump. That’s because in that situation discontinuity is between start and end position, so those positions are analogous to physical and mental state.

  Any brain mechanism, is still part of the physical. It’s true that there are some critical behaviors in the brain (similar to balls rolling down that hill). But the result of this criticality is still a physical state. So we cannot use a critical physical mechanism, to explain the discontinuity between physical and mental.

• Filip Sondej 2 Nov 2022 21:17 UTC

  1 point

  0

  in reply to: Alex Flint’s comment on: Mind is uncountable

  It just looks that’s what worked in evolution—to have independent organisms, each carrying its own brain. And the brain happens to have the richest information processing and integration, compared to information processing between the brains.

  I don’t know what would be necessary to have a more “joined” existence. Mushrooms seem to be able to form bigger structures, but they didn’t have an environment complex enough to require the evolution of brains.

• Filip Sondej 2 Nov 2022 18:48 UTC

  1 point

  0

  in reply to: Alex Flint’s comment on: Mind is uncountable

  It seems that we just never had any situations that would challenge this way of thinking (those twins are an exception).

  This Cartesian simplification almost always works, so it seems like it’s just the way the world is at its core.