Jonathan Moregård

[Question] 2 (naive?) ideas for alignment

The Round Table Manifesto

[Question] Non-co­er­cive mo­ti­va­tion for al­ign­ment re­search?

• Jonathan Moregård 28 Mar 2022 11:10 UTC

  6 points

  on: Do a cost-benefit anal­y­sis of your tech­nol­ogy usage

  Does anyone know about an addon to filter facebook notifications? I want to know about comments, but not reactions/​likes

• Jonathan Moregård 6 Sep 2022 6:11 UTC

  3 points

  1

  on: The shard the­ory of hu­man values

  Regarding time inconsistency of rewards, where subjects displayed a “today-bias”, might this be explained by shards formed in relation to “payout-day” (getting pocket money or salary)? For many people, agency and well-being vary over the month, peaking on the day of their monthly payout. It makes sense to me that these variations create a shard that values getting paid TODAY rather than tomorrow.

  For the 365 vs 366 example, I would assume that the selection is handled more rationally, optimizing for the expected return.

• Jonathan Moregård 28 Oct 2022 13:01 UTC

  1 point

  0

  on: The Game of Antonyms

  Something in me doesn’t like putting love <-> disgust as antonyms.

  love to me can be abstracted to prioritizing the utility of others without regard for your own. (at least the agape kind of love). I’d put the antonym as exploitation.

  disgust to me is about seeing something as lower/​unclean. To me the antonym for disgust is reverence.

  I think this is a bit too diffuse to actually have correct answers. but I like playing with concepts (programmer), so thanks for the game.

Strat­egy of In­ner Conflict

• Jonathan Moregård 16 Nov 2022 8:42 UTC

  4 points

  0

  in reply to: noggin-scratcher’s comment on: Strat­egy of In­ner Conflict

  I’ve fixed the spelling, thanks for the correction

The Echo Principle

• Jonathan Moregård 17 Nov 2022 9:20 UTC

  1 point

  0

  in reply to: dreeves’s comment on: Strat­egy of In­ner Conflict

  Didn’t expect this reply, thanks for taking your time. I do mention Beeminder briefly at one point, and yes, a lot of the post is about how beeminder-esque motivational strategies tend to backfire.

  To start with: I have friends that thrive on coercive motivational strategies. I’m pretty sure my claims aren’t universally applicable. However, coercive approaches seems to be a strong cultural norm, and a lot of people use coercive strategies in unskillful ways (leading to procrastination etc). These people might find a lot of value in trying out non-coercive motivational strategies.

  Reading your linked pages, I start thinking about what makes coercive motivations (or “self-discipline”, as you write on your page) a good fit for some and a bad fit for others. Might write up something about that on my substack in the future, I’ll link it to LW if I remember. Also, I’m curious is there a pre/​trans dynamic here, where non-coercion after coercison is different to non-coercion from the beginning.

  As for your concrete claims:

  What are the “smart, specific ideas” I suggested? In this post I mainly attempted to describe what not to do, and ended with some basic non-coercion. I’m curious what you found valuable.

  Re: bare minimum that would be irrational to fall below/​insurance. Maybe this is correct! I think I would find it hard to mix strategies in this way, since coersion vs non-coersion are pretty far apart as paradigms. A lot of the difference is about how you view yourself. I’m concerned that the coercion might “leak” through, if you keep it as a plan B. But then again, I haven’t thought about this, so take it with a pinch of salt :)

  Re: CBT & “Conflict vs Cooperation” (I interpret as coersion vs non-coersion). This feedback really tickled my nerd-spot. I’m a practicing stoic, and CBT is basically stoicism without the ontologies/​eudaimonia. In my mind, CBT/​Stoicism is about shifting personality traits and behavior patterns through changing actions, judgements and thought patterns. These are interconnected, in just the way you’re saying, and I agree that it’s possible to bootstrap new thought patterns by changing one’s actions.

  However, this is orthogonal to my post. I’m not claiming that coercive motivational strategies are bad because they are “shallow”, I’m claiming that they are bad because they lead to unnecessary friction, and might be full-out counter-productive since it’s easy to misuse and act in unskillful ways. The “it doesn’t affect fundamental things, we need to be holistic” is a common critique of CBT therapy as well, and I always find it ironic. I find it ironic because the critique assumes it’s possible to shift actions without affecting personality, which is a non-holistic perspective on the psyche. Hoisted by their own petard.

• Jonathan Moregård 17 Nov 2022 19:50 UTC

  6 points

  0

  on: Deon­tol­ogy and virtue ethics as “effec­tive the­o­ries” of con­se­quen­tial­ist ethics

  I really enjoyed your “successor agent” framing of virtue ethics! There are some parts of the section that could use clarification:

  Virtue ethics is the view that our actions should be motivated by the virtues and habits of character that promote the good life

  This sentence doesn’t make sense to me. Do you mean something like “Virtue ethics is the view that our actions should be motivated by the virtues and habits of character they promote” or “Virtue ethics is the view that our actions should reinforce virtues and habits of character that promote the good life”? It looks like two sentences got mixed up.

  “Virtues are not intrinsically right or wrong;”

  I get confused by this statement. I think of virtue ethics as putting all moral value onto the way you are training yourself to act. Virtue is the sole Good etc. Can you clarify what you mean here?

  “Taking honesty as an example virtue, we should strive to be honest, even if being dishonest would lead to some greater good”

  I guess you mean “lead to consequences that would be better according to a consequentialist perspective”. When discussing different views on ethics the term “good” gets overloaded.

• Jonathan Moregård 18 Nov 2022 6:06 UTC

  2 points

  0

  on: AGIs may value in­trin­sic re­wards more than ex­trin­sic ones

  But even if so, we (along with many other non-human animals) seem to enjoy and receive significant fulfillment from many activities that are extremely unlikely to lead to external rewards (e.g. play, reading etc).

  I see play serving some vital functions:

  1. exploring new existential modes. Trying out new ways of being without having to take a leap of faith.

  2. connecting with people, and building trust. I include things like flirting, banter, and make-believe here.

  As for reading, I think of it as a version of exploring.

  Note that there are certain behaviours that I’m sure aren’t very adaptive, but I have a hunch that many of them can be traced back to some manner of fitness improvement. My current hunch (pinch of salt please) is that most seemingly unnecessary action-categories either serve a hidden purpose , or are “side effects”. By “side effects”, I mean that the actions & habits spring from a root shared with other (more adaptive) behaviour patterns. This “root” can be a shard residing at a high abstraction level, or some instinct, depending on your view.

  Also, as I’m writing this, I realize that this is very hard to falsify and that my claims aren’t super rigorous. Hope it can be of some use to someone anyway.

• Jonathan Moregård 27 Dec 2022 14:07 UTC

  3 points

  0

  on: How evolu­tion­ary lineages of LLMs can plan their own fu­ture and act on these plans

  Does it make sense to ask AI orgs to not train on data that contains info about AI systems, different models etc? I have a hunch that this might even be good for capabilities: feeding output back into the models might lead to something akin to confirmation bias.
  
  Adding a filtering step into the pre-processing pipeline should not be that hard. Might not catch every little thing, and there’s still the risk of stenography etc, but since this pre-filtering would abort the self-referential bootstrapping mentioned in this post, I have a hunch that it wouldn’t need to withstand stenography-levels of optimization pressure.
  
  Hope I made my point clear, I’m unsure about some of the terminology.

• Jonathan Moregård 27 Dec 2022 18:17 UTC

  3 points

  0

  in reply to: Roman Leventov’s comment on: How evolu­tion­ary lineages of LLMs can plan their own fu­ture and act on these plans

  I don’t see how excluding LW and AF from the training corpus impacts future ML systems’ knowledge of “their evolutionary lineage”. It would reduce their capabilities in regards to alignment, true, but I don’t see how the exclusion of LW/​AF would stop self-referentiality.
  
  The reason I suggested excluding data related to these “ancestral ML systems” (and predicted “descendants”) from the training corpus is because that seemed like an effective way to avoid the “Beliefs about future selves”-problem.
  
  I think I follow your reasoning regarding the political/​practical side-effects of such a policy.
  
  Is my idea of filtering to avoid the “Beliefs about future selves”-problem sound?
  (Given that the reasoning in your post holds)
  
  

• Jonathan Moregård 27 Dec 2022 18:30 UTC

  2 points

  0

  in reply to: Roman Leventov’s comment on: How evolu­tion­ary lineages of LLMs can plan their own fu­ture and act on these plans

  Another question (that might be related to excluding LW/​AF):

  This paragraph:

  Consequently, the LLM cannot help but also form beliefs about the future of both “selves”, primarily the “evolutionary” one, at least because this future is already discussed in the training data of the model (e. g., all instances of texts that say something along the lines of “LLMs will transform the economy by 2030”)

  Seems to imply that the LW narrative of sudden turns etc might not be a great thing to put in the training corpus.

  Is there a risk of “self-fulfilling prophecies” here?

• Jonathan Moregård 27 Dec 2022 20:30 UTC

  1 point

  0

  in reply to: Roman Leventov’s comment on: How evolu­tion­ary lineages of LLMs can plan their own fu­ture and act on these plans

  In your other post, you write:

  “However, I’m very sceptical that this will happen in chat batch agents (unless developers “conveniently” indicate training and deployment using a special tag token in the beginning of the prompt!) because they are trained on the dialogues in the internet, including, presumably, dialogues between an older version of the same chat batch agent and its users, which makes it impossible to distinguish training from deployment, from the perspective of a pure language model.”

  This seems like a potential argument against the filtering idea, since filtering would allow the model to disambiguate between deployment and training.

• Jonathan Moregård 27 Dec 2022 20:38 UTC

  1 point

  0

  on: Prop­er­ties of cur­rent AIs and some pre­dic­tions of the evolu­tion of AI from the per­spec­tive of scale-free the­o­ries of agency and reg­u­la­tive development

  I struggle following the section “Bigger boundaries mean coarse-graining”. Is there a way to express it in non-teleologic language? Can you recommend any explainers or similar?

• Jonathan Moregård 4 Jan 2023 14:00 UTC

  10 points

  2

  on: Knottiness

  This is very related to Radical Honesty, part of the authentic relating movement. The basic idea is that by being extremely honest, you connect more with other people, let go of stress induced by keeping track of narratives, and start realizing the ways in which you’ve been bullshitting yourself.

  When I started, I discovered a lot of ways in which I’d been restricting myself with semi-conscious narratives, particularly in social & sexual areas of life. Expressing the “ugh” allowed me to dissolve it more effectively.

• Jonathan Moregård 6 Jan 2023 10:23 UTC

  1 point

  0

  on: Things I carry al­most ev­ery day, as of late De­cem­ber 2022

  Regarding supplements: consider using some kind of pill organizer instead of carrying around the entire containers.
  
  Something like:
  https://​​www.amazon.com/​​EZY-DOSE-Organizer-Medicine-Compartments/​​dp/​​B0000532OS/​​ref=sr_1_10?crid=1YVWSL9GM3KOW&keywords=7-day+organizer&qid=1673000500&sprefix=7-day+orghanizer%2Caps%2C162&sr=8-10

  or

  
  https://​​www.amazon.com/​​gp/​​product/​​B07ZV1P83W/​​ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1

• Jonathan Moregård 9 Jan 2023 21:04 UTC

  5 points

  2

  on: The Foun­tain of Health: a First Prin­ci­ples Guide to Rejuvenation

  This is very interesting. “We should increase healthspans” is a much more palatable sentiment than “Let’s reach longevity escape velocity”. If it turns out healthspan aligns well with longevity, we don’t need to flip everyone’s mindsets about the potential for life extension; we can start by simply pointing to interventions that aim to mitigate the multi-morbidity of elderly people.

  “Healthy ageing” doesn’t disambiguate between chronological age and metabolic health the way you try to do in this post, but it can still serve as a sentiment that’s easy to fit inside the Overton window.