martinkunev

[Question] Would you have a baby in 2024?

How use­ful is Cor­rigi­bil­ity?

• martinkunev 5 Sep 2023 22:11 UTC

  7 points

  on: Eliezer Yud­kowsky Facts

  Eliezer Yudkowsky once entered an empty newcomb’s box simply so he can get out when the box was opened.
  
  or
  
  When you one-box against Eliezer Yudkowsky on newcomb’s problem, you lose because he escapes from the box with the money.

[Question] Choos­ing a book on causality

What is On­tol­ogy?

Dis­in­cen­tiviz­ing de­cep­tion in mesa op­ti­miz­ers with Model Tampering

• martinkunev 19 Feb 2024 17:30 UTC

  3 points

  2

  in reply to: Liron’s comment on: Goal-Com­plete­ness is like Tur­ing-Com­plete­ness for AGI

  I find the ideas you discuss interesting, but they leave me with more questions. I agree that we are moving toward a more generic AI that we can use for all kinds of tasks.

  I have trouble understanding the goal-completeness concept. I’d reiterate @Razied ’s point. You mention “steers the future very slowly”, so there is an implicit concept of “speed of steering”. I don’t find the turing machine analogy helpful in infering an analogous conclusion because I don’t know what that conclusion is.

  You’re making a qualitative distinction between humans (goal-complete) and other animals (non-goal complete) agents. I don’t understand what you mean by that distinction. I find the idea of goal completeness interesting to explore but quite fuzzy at this point.

• martinkunev 16 Feb 2024 2:41 UTC

  3 points

  on: Test­ing The Nat­u­ral Ab­strac­tion Hy­poth­e­sis: Pro­ject Intro

  I’m wondering whether useful insights can come from studying animals (or even humans from different cultures) - e.g. do fish and dolphins form the same abstractions; do bats “see” using ecolocation?

• martinkunev 12 Feb 2024 2:11 UTC

  3 points

  1

  on: Stop Defeat­ing Akra­sia (1/​4)

  I hope the next parts don’t get delayed due to akrasia :)

• martinkunev 16 Jan 2024 20:58 UTC

  3 points

  0

  on: In­ner and outer al­ign­ment de­com­pose one hard prob­lem into two ex­tremely hard problems

  @TurnTrout You wrote “if I don’t, within about a year’s time, have empirically verified loss-as-chisel insights which wouldn’t have happened without that frame...”

  More than a year later, what do you think?

• martinkunev 6 Jan 2024 0:02 UTC

  3 points

  2

  in reply to: RogerDearnaley’s comment on: The al­ign­ment sta­bil­ity problem

  I tend to agree but I believe most non-aligned behavior is due to scarcity. It’s hard to get into the heads of people like Stalin but I believe if everybody has a very realistic virtual reality where they could do all the things they’d do in real life, they may be much less motivated to enter into conflict with other humans.

• martinkunev 1 Nov 2023 1:12 UTC

  3 points

  on: Writ­ing Causal Models Like We Write Programs

  The letters I and l look the same. Maybe use 1 instead of upper case i?

• martinkunev 15 Oct 2023 13:19 UTC

  3 points

  on: Study Guide

  The Games and Information book link is broken. It appears to be this book:
  https://​​www.amazon.com/​​Games-Information-Introduction-Game-Theory/​​dp/​​1405136669/​​ref=sr_1_1?crid=2VDJZFMYT6YTR&keywords=Games+and+Information+rasmusen&qid=1697375946&sprefix=games+and+information+rasmuse%2Caps%2C178&sr=8-1

• martinkunev 28 Mar 2023 22:57 UTC

  3 points

  0

  on: What Are You Track­ing In Your Head?

  When outside, I’m usually tracking location and direction on a mental map. This doesn’t seem like a big deal to me but in my experience few people do it. On some occasions I am able to tell which way we need to go while others are confused.

• martinkunev 19 Mar 2024 11:12 UTC

  2 points

  0

  on: Anx­iety vs. Depression

  Both gravity and inertia are determined by mass. Both are explained by spacetime curvature in general relativity. Was this an intentional part of the metaphor?

• martinkunev 12 Nov 2023 2:51 UTC

  2 points

  0

  in reply to: DaemonicSigil’s comment on: There are no co­her­ence theorems

  “paying to avoid being given more options looks enough like being dominated that I’d want to keep the axiom of transitivity around”
  
  Maybe offtopic but paying to avoid being given more options is a common strategy in negotiation.

• martinkunev 29 Oct 2023 12:02 UTC

  2 points

  0

  on: Utili­tar­i­anism Meets Egalitarianism

  “However, when everyone gets expected utility 1, the expected logarithm of expected utility will have the same derivative as expected expected utility”

  
  Can you clarify this sentence? What functions are we differentiating?

• martinkunev 1 Oct 2023 23:20 UTC

  2 points

  0

  on: My im­pres­sion of sin­gu­lar learn­ing theory

  To make this easier to parse on the first read, I would add that

  N is the number of parameters of the NN and we assume each parameter is binary (instead of the usual float).

• martinkunev 26 Jul 2023 1:49 UTC

  2 points

  on: ToL: Foundations

  The image doesn’t load.

  The notation in Hume’s Black Box seems inconsistent. When defining [e], e is an element of a world. When defining I, e is a set of worlds.

• martinkunev 31 Mar 2023 17:30 UTC

  2 points

  0

  on: My Ob­jec­tions to “We’re All Gonna Die with Eliezer Yud­kowsky”

  “discovering that you’re wrong about something should, in expectation, reduce your confidence in X”

  This logic seems flawed. Suppose X is whether humans go extinctinct. You have an estimate of the distribution of X (for a bernoulli process it would be some probability p). Take the joint distribution of X and the factors on which X depends (p is now a function of those factors). Your best estimate of p is the mean of the joint distribution and the variance measures how uncertain you’re about the factors. Discovering that you’re wrong about something means becoming more uncertain about some of the factors. This would increase the variance of the joint distribution. I don’t see any reason to expect the mean to move in any particular direction.

  Or maybe I’m making a mistake. In any case, I’m not convinced.