Jonas Hallgren

[Question] Is there any se­ri­ous at­tempt to cre­ate a sys­tem to figure out the CEV of hu­man­ity and if not, why haven’t we started yet?

• Jonas Hallgren 17 May 2021 10:00 UTC

  1 point

  in reply to: Dagon’s comment on: Con­cern­ing The Ul­ti­mate Mo­ral The­ory—Rev­ersible Utilitarianism

  Good question, this is rather applied on a system scale level so for example, a democratic system is going to be inherently more reversible than a non-democratic system. An action that goes against the reversibility of a system could for example be the removal of freedom of speech as it would narrow down the potential pathways of future civilizations. Reversibility has an opportunity cost inherent to it as it asks us to take into consideration the possibility of other morals being correct. This is like Pascal’s mugging but with the stakes that if we have the wrong moral theory then we lose a lot. This means that if you have a utilitarian lens it might be less effective as there are actions that might be good from the utilitarian standpoint such as turning everything into hedonium, that are bad from a reversibility standpoint as we can’t change anything from there.

• Jonas Hallgren 19 May 2021 11:35 UTC

  3 points

  on: Sav­ing Time

  But the problem runs deeper than that. If we draw an arrow in the direction of the deterministic function, we will be drawing an arrow of time from the more refined version of the structure to the coarser version of that structure, which is in the opposite direction of all of our examples.

  As I currently understand this after thinking about it for a bit, we are talking about the coarseness of the model from the perspective of the model in the timeframe that it is in and not the time frame that we are in. It would make sense for our predictions of the model to become more coarse with each step forward in time if we are predicting it from a certain time into the future time-space. I don’t know if this makes sense but I would be grateful for a clarification!

[Question] Is it worth mak­ing a database for moral pre­dic­tions?

• Jonas Hallgren 1 Sep 2021 15:08 UTC

  1 point

  on: [Spon­sored] Job Hunt­ing in the Modern Economy

  Wow, this changed my life! Never thought I would find something this mind-blowingly overpowered on LessWrong!

• Jonas Hallgren 23 Sep 2021 8:52 UTC

  3 points

  on: Where do your eyes go?

  Maybe this isn’t the most productive comment but I just wanted to say that this was a really good post. It’s right down my alley with video games and academics at the same time and I would therefore like to declare it a certified hood classic. (apparently Grammarly thinks this comment is formal which is pretty funny.)

• Jonas Hallgren 26 Oct 2021 11:22 UTC

  4 points

  on: My ex­pe­rience at and around MIRI and CFAR (in­spired by Zoe Curzi’s writeup of ex­pe­riences at Lev­er­age)

  It feels kind of weird that this post only has 50 upvotes and is hidden in the layers of lesswrong as some skeleton in the closet waiting to strike at an opportune time. A lot of big names commented on this post and even though it’s not entirely true and misrepresenting what happened to an extent it would make sense to kind of promote this type of a post anyway. It’s setting a bad example if we don’t promote as we then show that we don’t encourage criticism which seems very anti-rational. Maybe a summary article of this incident could be done and put on the main website? It doesn’t make sense to me that a post with a whooping 900 comments should be this hidden and it sure doesn’t look good from an outside perspective.

[Question] Does agent foun­da­tions cover all fu­ture ML sys­tems?

An­nounc­ing the Distil­la­tion for Align­ment Practicum (DAP)

• Jonas Hallgren 18 Aug 2022 23:51 UTC

  1 point

  0

  in reply to: ThomasW’s comment on: An­nounc­ing the Distil­la­tion for Align­ment Practicum (DAP)

  First and foremost, my confidence in the descriptions of different distillation methods is pretty low. It is a framework I’ve thrown together from discussions on what an optimal science communication landscape would look like. It is in its initial phases and will most likely be imperfect for quite some time as finding the optimal communication landscape is a difficult problem.
  
  Secondly, Great point! I think that my thinking of it, is as a “reinterpretation of existing research.” The basic way of doing this is rewriting a post for higher clarity which is the classical way that a distillation is viewed from.
  
  I think there are more ways of doing this and that the space is underexplored. In terms of the terminology proposed in the course, a “classic” distillation is some combination of what I would describe as propagating and bushwacking.
  
  Bushwacking would be more something like asking, “what the f*ck is going on here?” which might be relevant for things such as infra-bayesianism (I want to learn infra-bayesianism can someone please bushwack this).
  
  Propagating would be more of what Rob Miles is doing.
  
  So what is distillation? What is the superclass of all of these?

  I would phrase it like the following “A distillation is a work that takes existing research and reinterprets it in a new light.”
  

  Finally, a meta point in defence of the introduction of new jargon. I think the term distillation is confusing in itself as it can mean a lot of things, and therefore if you say, “I’m bushwhacking this post” you get the idea that “ah, this person is cutting down the weeds of what is a confusing post”. I hope to introduce new methodology so it is easier to understand what type of distillation someone is doing. (I don’t think this terminology is optimal, but it’s a start in the right direction IMO.)

• Jonas Hallgren 30 Aug 2022 1:44 UTC

  4 points

  1

  on: An In­tro­duc­tion to Cur­rent The­o­ries of Consciousness

  I wanted to say that I tried learning IIT about 2 years ago after reading about most other theories of consciousness and that it was a pain in the ass and so I gave up. Thank you for this post and especially that section. I really like attention schema theory as I hadn’t thought about combining the approaches of GWT and strange loops.
  
  I’ve also got one thing that I want to bring up about your conclusion and also one perspective that I personally find interesting that you don’t necessarily bring up in the post.
  
  With regards to the conclusion:
  Why do you assume that consciousness is an illusion just because consciousness is a strange loop observing itself? Why can’t self-referential things be real? My belief here is that everything that is a possible mathematical structure is real, and even though self-referring leads to error in any axiomatic system, that just means we can’t define the things themselves. Just like how spacetime forms singularities, this can exist in a world which is as real as any other world.
  
  Secondly, you mention panpsychism quickly, but you don’t mention the no-self perspective and panpsychism. This is quite an eastern perspective and states that you’re every experience that you have and that everything that appears is itself part of consciousness. This is essentially panpsychism, with every experience divided into it’s smallest subcomponents. The evidence for this would be meditation, I can feel that I am the sense experience in my fingers while writing. You don’t really mention this view.
  
  Lastly, I want to mention how I bring these two views together in my head. My feeling of being my fingers doesn’t arise from the fact that I’m observing myself as a strange loop but instead that every experience is a strange loop in itself.
  The logical conclusion of strange loops is in my opinion that every part of reality is a strange loop viewing itself and that every system can come up with a symbol for I even if it’s not what we think of as thinking.

• Jonas Hallgren 16 Oct 2022 22:27 UTC

  11 points

  8

  on: Age changes what you care about

  (PSA:)
  Hey you, whoever is reading this comment, this post is not an excuse to skip working on alignment. I can fully relate to the fear of death here, and my own tradeoff is focusing hard on instrumental goals such as my own physical health and nutrition (including supplements) to delay death and get some nice productivity benefits. This doesn’t mean that an AI won’t kill you within 15 years, so it’s most likely not even a defect in a tragedy of commons to not work on it; it’s rather paramount to your future success at being alive.
  
  (Also if we solve alignment, then we can get a pretty op AGI that can help us out with the other stuff so really it’s very much a win-win in my mind.)

• Jonas Hallgren 7 Jan 2023 12:38 UTC

  3 points

  0

  on: Run­ning by Default

  I just wanted to make a comment that I deliberately choose to not run to allow my daydreaming mode (default mode network) to become active and to “recharge” my deep work system. Basically, there is evidence that walking makes you more creative, and it’s a really easy way for me to get into that specific brain state. I also use it as a shorter idea-based meditation so that I can focus on a problem and try to solve it whilst walking.

• Jonas Hallgren 21 Feb 2023 14:32 UTC

  1 point

  −1

  on: Pod­cast: What’s Wrong With LessWrong

  I’m just going to write a review of this for anyone wants to get to the meat of the critique. I would consider myself outside the drama part of this as I’ve basically only engaged with the ideas and not that much with the “community”. So I won’t go into any of the community drama stuff but mostly factual disagreements.

  (My take will of course still be biased however.)

  General: I think Alfred did this in reasonably good faith most of the time. I think there was stuff he definetely skipped engaging with but I was honestly thinking it would be more drama and hit-piecy than it was.

  TL;DR (of good points):

  Suboptimal models of cognition when it comes to who can do good work as LW tends to care more about G then it does about for example conscientiousness and creativity that should be prioritised higher.

  LessWrongers tend to lose the root for the tree when it comes to self optimisation. Exercise more and start thinking about the underlying cognitive algorithms of getting stuff done in the world.

  There are weird culture norms where privacy and a homogenous population leads to in-group thinking and biases towards rediscovering things that already exist in the world within the LW sphere.

  On different time stamps:

  0:53 most of the sequences aren’t about rationality; AI is not rationality

  Death is bad is all i say here.

  3:43 lesswrong and IQ mysticism

  A bit handwavy but points to misunderstandings in how people actually make stuff in the world and the cult of genius.

  32:20 lesswrong and something-in-the-waterism

  Claim: you can do most things online anyway, why go to the Bay? //​This seems like an intially good argument but there are major serendipity effects in terms of encountering new ideas that I feel he doesn’t bring up. He doesn’t bring up arguments such as the modes of cognition being more variable as you can go on a walk or have an in-person discussions with people.

  36:49 overtrusting of ingroups

  Summary: LW is way too in-group with too little mechanisms that can allow for calling out BS.

  39:35 vulnerability to believing people’s BS self-claims

  Summary: People lie about their own capabilities in terms of for example writing speed which is bad.

  47:35 norms aren’t sharp enough

  Summary: people aren’t calling out bullshit. Points to something like “you sound ridicolous”. An example is apparently saying “I drink soylent because it is more efficient” and this getting no pushback. //​Yet pushback is based on if you share beloefs or not and so it seems this is more based on personal disagreements.

  {54:41 weird cultlike privacy norms

  56:46 realnaming as “doxxing”

  58:28 no viable method for calling out rumors/​misinformation if realnaming is ‘doxxing’

  1:00:16 the strangeness and backwardness of LW-sphere privacy norms}:

  Summary: Privacy norms are weird in LW. //​might be true, uncertain.

  {1:04:07 EA: disregard for the homeless and refusal to do politics because it’s messy

  1:10:16 EA: largely socially inept, does not understand how truly bad the SBF situation is

  1:13:36 EA: treatment of utilitarianism and consciousness is simplistic

  1:20:20 EA rigor: vitamin A charity example}:

  Vibe: “I disagree with longtermism and animal suffering and so it’s bad” //​I know this isn’t charitable but it is a low level on the discussion here so I will respond in the same way. He also makes some goof points about QALY’s and perception of pain to counteract utilitarianism not that deep of a discussion but still pretty good points.

  1:23:39 extreme techno optimism and weak knowledge of human biology

  //​I would tend to agree that LWs could exercise more and learn more about neurobiology.

  1:25:24 exclusionary white nerd millennial culture

  Summary: homogenous culture

  1:27:23 comfort class culture

  Summary: sheltered and upper-middle class creates sheltering and implicit culture norms

  1:30:25 pragmatics-agnosticism

  Summary: yes you may be rational but what about writing and tonation and other parts of life?

  1:33:13 shallow analysis of empirical topics

  1:34:18 idiosyncrasies of communication, e.g. being extremely obtuse at the thesis level

  Summary: People don’t listen that well on LW. The letter of an argument is being followed rather than the spirit.

  1:39:50 epistemic rationality matters much more than instrumental rationality

  1:43:00 the scene isn’t about rationality, it’s about hanging out and board games (which is fine, just don’t act like you’re doing anything important)

  Summary: Some epic ranting about how LW is about talking about AI and since he doesn’t believe the AI stuff people are belittled to “hanging out and playing board games”.

  If you cba with implicit moral disagreements (such as the ones that underly the very epic EA drama that keeps popping up (not saying it’s only that on the EA forum)) I would recommend thinking “I should exercise more and be more careful with implicit group norms in the future” and move on with your life.

• Jonas Hallgren 21 Feb 2023 15:19 UTC

  1 point

  0

  in reply to: alfredmacdonald’s comment on: Pod­cast: What’s Wrong With LessWrong

  Latest time I read the literature (1-2 years ago) this was also my conception. If I remember correctly then the predictability of nobel price winners based on their IQ was very low conditional on if they had above 130 IQ. I think conscientousness and creativity were described generally more predictive for gettibg a nobel prize at higher IQs.

  (fyi; creativity is quite hard to measure as it is a complex topic. Huberman has a great episode on the mechanisms behind creativity from December)

  I do however, also want to mention that there probably exists a “package” that makes someone very capable where genius is part of it. I just think that IQ is overhyped when it comes to predicting this.

Power-Seek­ing = Min­imis­ing free energy

• Jonas Hallgren 1 Mar 2023 1:54 UTC

  3 points

  0

  in reply to: avturchin’s comment on: Power-Seek­ing = Min­imis­ing free energy

  I think this question mainly points towards the weird interaction of this with objectives and goals.
  
  If we set the survival of offspring as the goal, then when we die, we will have no power over the future. This is the same thing as us having max entropy or chaos over the future. So yeah escaping death = minimising free energy with respect to goals that require action in the future.

  When we minimise the external free energy, we do it over all future time and not only our current timestep.

  We don’t only care about the universe until our current point in time, but we do so for all the future. We’re trying to align our model of the world so that all potential timelines have as little external chaos in them as possible. (Incidentally, this is the same thing as minimising the number of potential timelines.)

  

• Jonas Hallgren 1 Mar 2023 2:23 UTC

  1 point

  0

  in reply to: Dan H’s comment on: Power-Seek­ing = Min­imis­ing free energy

  I understand how that is generally the case, especially when considering evolutionary systems’ properties. My underlying reason for developing this is that I predict using ML methods on entropy-based descriptions of chaos in NNs will be easier than looking at pure utility functions when it comes to power-seeking.
  
  I imagine that there is a lot more work on existing methods for measuring causal effects and entropy descriptions of the internal dynamics of a system.
  
  I will give an example as the above seems like I’m saying “emergence” as an answer to why consciousness exists, it’s non-specific.

  If I’m looking at how deception will develop inside an agent, I can think of putting internal agents or shards against each other in some evolutionary tournament. I don’t know how to set up an arbitrary utility for these shards, so I don’t know how to use the evolutionary theory here. I do know how to set up a potential space of the deception system landscape based on a linear space of the significant predictive variables. I can then look at how much each shard is affecting the predictive variables and then get a prediction of what shard/​inner agent will dominate the deception system through the level of power-seeking it has.

  Now I’m uncertain whether I would need to care about the free energy minimisation part of it or not. Still, it seems to me that it is more useful to describe power-seeking and what shard/​inner agent ends up on top in terms of information entropy. (I might be wrong and if so I would be happy to be told so.)

• Jonas Hallgren 1 Mar 2023 3:26 UTC

  1 point

  0

  on: My Ex­pe­rience With Lov­ing Kind­ness Meditation

  I just wanted to write down a relation to some interesting resources, which is some of QRI’s research ideas. They had an idea that there is one scale of intensity and one scale of valence when it comes to experience. When you take specifically psychedelics or meditate, it is like you’re increasing the intensity of your mental states or in other words, the power of your brain. Where you end up on the valence scale is dependent on the initial conditions (for example, whether you’re stressed out or not). I think this can be a helpful model as to why meditation can come to backfire from time to time; by increasing the power of the mind, you risk states that you couldn’t access before.
  
  Now, this is quite explorative science in itself, but it matches pretty well with existing models.

  Anyways, I got a big smile on my face and metta in my system when you mentioned feeling metta for your toddler, so thank you for sharing and have a wonderful day!

• Jonas Hallgren 4 Mar 2023 4:51 UTC

  1 point

  0

  on: Acausal normalcy

  So I’ve got a hypothetical model for how boundary-breaking can be measured in terms of the predictability of world models of the future.
  
  The hypothesis is that the higher the effect of a system with a boundary on the predictability of your future world models, the higher the reward prediction error (or whatever equivalent measure you want to use) is if the boundary is broken.
  
  Example: If you believe that your community are all focused and working together towards one common goal, then you are willing to give a lot away to other people if you believe the community will socially reciprocate. When the boundary of social reciprocity is broken, the future worlds where you were counting on reciprocity being intact turn negative in EV compared to before, and reward prediction error happens on your prediction of future worlds.

  The higher the importance of the boundary-breaking, the higher the change in potential worlds and EV. This in turn means higher reward prediction error.