Viktor Rehnberg

Karma: 152

Viktor Rehnberg 11 Jun 2021 11:16 UTC
14 points
on: Would you like me to debug your math?
I am one of those people that have benefited from working with Gurkenglas this way.
For me in particular the main benefits have been two-fold.
1. Becoming more aware of short comings in how I program. Especially noticing when something I’m doing a lot has a better solution, but I haven’t gotten round to investigating alternatives because each individual time is faster to do the old way rather than learning a new way. Gurkenglas helped me get around to actually doing something about this.
2. Math. It could be when you are doing something that feels too ad-hoc, Gurkenglas can help you work out a more canonical approach.
Now if I had seen this post before I had worked with Gurkenglas I would most probably not have reached out for mainly two reasons
1. Reaching out to strangers is hard and I might not have felt that what I was doing would be worth taking up a whole other persons time.
2. I would probably not know what I was missing. I would think that the way I was doing things were fine and there would be little to benefit from it.
If you are thinking something similar, I’m here to tell you that you should probably try it out anyway. The risk of trying and finding that it wasn’t worth it is probably much lower than the risk of not trying and missing some good level up opportunities.

Viktor Rehnberg 11 Jun 2020 6:43 UTC
14 points
on: I’m leaving AI alignment – you better stay
As someone just starting out on the path towards becoming AI safety researcher I appreciate this post a lot. I have started worrying about not having enough impact in the field if I could not become established fast enough. However, reading this I think that it might serve me (and the field) better if I instead take my time and instead only enter the field properly if I find that my personal fit seems good and that I can stay in the field for a long time.
Furthermore, this post has helped me in finding possible worthwhile early on projects that could increase my understanding and personal fit for the field.

Viktor Rehnberg 1 Jul 2020 10:09 UTC
10 points
0
on: Occam’s Razor
My own way of thinking of Occam’s Razor is through model selection. Suppose you have two competing statements $H_{1}$ (the which did it) and $H_{2}$ (it was chance or possibly something other than a which caused it ( $H_{2} = \neg H_{1}$ )) and some observations $D$ (the sequence came up 0101010101). Then the preferred statement is whichever is more probable calculated as
$p (H | D) = \frac{p (H) p (D | H)}{p (D)}$
this is simply Bayes rule where
$p (D | H) = \int_{θ} p (D | θ, H) p (θ | H) d θ$
and the model is parametrized by some parameters $θ$ .
Now all this is just the mathematical way of writing that a hypothesis that has more parameters (or more specifically more possible values that it predicts), will not be as strong a statement that predicts a smaller state of outcomes.
In the witch example this would be:
- $H_{1} =$ There exist an advanced intelligent being (at least not much less than human intelligence) that can do things beyond what has ever been reproduced in a scientific way that for some reason chooses to live on our street and act mostly as a human that will choose to influence my sequence coin tosses to end up in some seemingly looking pattern
- $H_{2} =$ The coin toss is ruled by chance and might end up in the set of possible outcomes that seem to form a pattern ( ${0101010101, 1111100000, 1100110011, \dots}$ )
- $D =$ The coin toss ended up as $0101010101$
- The way I stated the hypotheses $p (D | H_{1}) = p (D | H_{2}) \cdot Fraction of outcomes that look like a pattern$
Now what remains is to estimate the priors and the the fraction of outcomes that look like a pattern. We can skip $p (D)$ as we are interested in $p (H_{1} | D) : p (H_{2} | D)$ .
Now comparing the amount of conditionals in the hypotheses and how surprised I am by them I would roughly estimate a ratio of the priors as something like $2^{100}$ in favor to chance, as the witch hypothesis goes against many of my formed beliefs of the world collected over many years, it includes weird choices of living for this hypothetical alien entity, it picks out me as a possible agent of many in the neighborhood, it singles out an arbitrary action of mine and an arbitrary set of outcomes.
For the sake of completeness. The fraction of outcomes that look like a pattern is kind of hard to estimate exactly. However, my way of thinking about it is how soon in the sequence would I postulate the specific sequence that it ended up in. After 0101, I think that the sequence 0101010101 is the most obvious pattern to continue it in. So roughly this is six bits of evidence.
In conclusion, I would say that the probability of the witch hypothesis is lacking around 94 bits of evidence for me to believe it as much as the chance hypothesis.
The downside of this approach to the Solomonoff induction and the minimum message length is that it is clunkier to use and it might be easy to forget to include conditionals or complexity in the priors the same way they can be lost in the English language. The upside is that as a model it is simpler, less ad hoc and builds directly on the product rule in probability and that probabilities sum to one and should thus be preferred by Occam’s Razor ;).

Viktor Rehnberg 31 Mar 2023 6:31 UTC
7 points
0
on: Some 2-4-6 problems
I knew about 2-4-6 problem from HPMOR, I really like the opportunity to try it out myself. These are my results on the four other problems:

indexA

Number of guesses:

8 guesses of which 3 were valid and 5 non-valid

Guess:

“A sequence of integers whose sum is non-negative”

Result: Failure

indexB

Number of guesses:

39 of which 23 were valid 16 non-valid

Guess:

“Three ordered real numbers where the absolute difference between neighbouring numbers is decreasing.”

Result: Success

indexC

Number of guesses:

21 of which 15 were valid and 6 non-valid

Guess:

“Any three real numbers whose sum is less than 50.”

Result: Success

indexD

Number of guesses:

16 of which 8 were valid and 8 non-valid

Guess:

“First number is a real number and the other two are integers divisible by 5”

Result: Failure

Performance analysis

I’d say that the main failure modes were that I didn’t do enough tests and I was a very bad number generator. For example, in indexD

I made 9 tests to test my final hypothesis 4 of which were valid, that my guess and the actual rule would give the same result for these 9 tests if I were actually good at randomizing is very small.

I could also say that I was a bit naive on the first test and that I’d grown overconfident after two successive successes for the final test.

Viktor Rehnberg 19 Sep 2023 17:31 UTC
6 points
0
on: Anthropic’s Responsible Scaling Policy & Long-Term Benefit Trust
To me it feels like this policy is missing something that accounts for a big chunk of the risk.

While recursive self-improvement is covered by the “Autonomy and replication” point, there is another risk from actors that don’t intentionally cause large scale harm but use your system to make improvements to their own systems as they don’t follow your RSP. This type of recursive improvement doesn’t seem to be covered by any of “Misuse” or “Autonomy and replication”.

In short it’s about risks due to shortening of timelines.

Viktor Rehnberg 22 Jun 2020 7:17 UTC
6 points
on: Simulacra Levels and their Interactions
Nice write up, I believe I have a better grasp of simulacra levels after this post.
(Think this is missing 1-3 additional roles. Discussion question, what is The Idealist?)
I’ll take this as an exercise for the readers.
I’ll start with a definition of how I see The Idealist: The Idealist is someone with an ideal of how people should act to have the best consequences in the world. In the most simple case this could simply be someone that believes the truth to be most important and that everyone should stay in Level 1. However, this type of idealist could ironically be seen as a Level 2 move: “I’m telling the truth so that you will stay on Level 1 and tell the truth as well”.
It becomes more complicated when the implications of the ideal affects higher simulacra levels. Consider an idealist with the ideal that intelligence is the most important and that advanced communication and progressively higher level reasoning is the way to achieve this. This way you value the complexity arising in Level 4 and you want people to see your group as good and that they move to a higher level.
In common, the types of idealist I can imagine want to affect the map so that people act in accordance with their ideal (Level 2 move), but they also want their group to be perceived as the cool group and will say things that will make undecided people move towards their group (Level 4 move). Therefore I would say that The Idealist is a Level 2 + Level 4 player. Furthermore, The Idealist only says something if it improves both how others perceive the map and how other value the in group (relative to the out group).
I don’t know if this is roughly what you had in mind when you thought about The Idealist, but this is my stab at the problem.

Viktor Rehnberg 8 Apr 2024 7:14 UTC
5 points
0
in reply to: Ape in the coat’s comment on: Should you refuse this bet in Technicolor Sleeping Beauty?
I’d say $P (Tail | Wake-up) = 2 / 3$

Viktor Rehnberg 4 Apr 2024 15:43 UTC
5 points
1
on: Should you refuse this bet in Technicolor Sleeping Beauty?
The possible observer instances and their probability are:
- Heads 50 %
  Red room 25 %
  Blue room 25 %
- Tails 50 %
  Red room 50 % (On Monday or Tuesday)
  Blue room 50 % (On Monday or Tuesday)
If I choose a strategy “bet only if blue” (or equivalentely “bet only if red”) then expected value for this strategy is $(- 300) * 0.25 + 200 * 0.5 = 25$ so I choose to follow this strategy.
I don’t remember what halfer and thirder were or what position I consider to be correct.

Viktor Rehnberg 7 Jul 2022 15:19 UTC
5 points
2
on: When Giving People Money Doesn’t Help
There’s something very interesting in this graph. The three groups have completely converged by the end of the 180 day period, but the average bank balance is now considerably higher.
Wasn’t the groups selected for having currently low income? Shouldn’t we expect some regression towards the mean i.e. an increase in average bank balance? Was there any indication for if the observed effect was larger or smaller than expected?

Viktor Rehnberg 16 Jun 2022 12:23 UTC
5 points
on: FYI: I’m working on a book about the threat of AGI/ASI for a general audience. I hope it will be of value to the cause and the community
It seems like it could be worthwhile for you to contact someone in connection to AGI Safety Communications Initiative. Or at the very least check out the post I linked.

Viktor Rehnberg 19 Apr 2021 20:02 UTC
5 points
on: D&D.Sci April 2021: Voyages of the Gray Swan
I’ve put my attempt in my Github but I’ll try to summarise it here.
Initial thoughts:
Edit: Changed to Rot13 as I failed on spoiler tags
V fgnegrq ol ybbxvat ng gur qngn naq V svtherq gung gur qvfgevohgvbaf cebonoyl jrer dhvgr avpr. Sebz ybbxvat ng gur qngn gurer frrz gb or abguvat rfcrpvnyyl jrveq tbvat ba jvgu qverpgvba, zbagu be gvzr. Gurer ner n srj abgnoyr guvatf gung ner n ovg yrff avpr guna gur erfg.
Cvengrf ner ovzbqny naq gurer bar bs zbqrf ner erqhprq va vagrafvgl nsgre nccebkvzngryl 62 jrrxf. Jngre ryrzragnyf ner havsbez, juvyr zbfg bs gur erfg ybbxf yvxr gurl pbhyq or pncgherq ol Tnzzn qvfgevohgvba. Nf fhpu V nffhzrq gung V pbhyq zbqry rirelguvat nf n Tnzzn qvfgevohgvba, rkprcg cvengrf gung jrer n Ovabzvny bire gjb Tnzzn naq jngre ryrzragnyf gung jrer havsbez naq gura n Zhygvabzvny sbe jung rapbhagre.
Ng svefg V gevrq n shyy Onlrfvna nccebnpu gb zbqry sbe gur cnenzrgref ohg ZP fnzcyvat guvf tbg n ovg gebhoyrfbzr naq V qvqa’g jnag gb chg va gur gvzr gb qb vg cebcreyl. Vafgrnq V qvq n ebhtu ZYR rfgvzngr jrer V nffhzrq vaqrcraqrapr juvpu grpuavpnyyl qvqa’g ubyq nf gurl funer gur haxabja qngn naq V unir nebhaq 700 qngn cbvagf sebz gur haxabja ynoryf gung jrer hanyybpngrq. Arireguryrff, V abj unq fbzrguvat V pbhyq hfr gb frnepu sbe gur orfg fbyhgvba.
Gelvat vg bhg vg qbrfa’g ybbx tbbq, hfvat nyy gur vagreiragvbaf yrnqf gb fbzrguvat gung unf nebhaq 3 % punapr bs fheivivat gur phzhyngvir qnzntr sebz gra pbafgvghgvir gevcf. Vs guvf vf gur pnfr V jbhyq gel irel uneq abg gb tb.
Gur orfg nggrzcg V sbhaq (gubhtu gur engr jnf fb fznyy fb V qvqa’g trg n tbbq pregnvagl sbe guvf) jnf gb hfr funex ercryynag, cnl gur zrecrbcyr n gevohgr naq ohl 15 rkgen bnef sbe gur erfg bs gur tbyq. Gur erfhyg jnf gura
* Punapr bs fheiviny 0.25 %
* Nirentr qnzntr ba fheivivat fuvcf 77 %
* Tbyq yrsg 0
V unira’g purpxrq guvf gubhtu, fb gurer zvtug irel jryy or fbzr oht gung tvirf zr jebat erfhygf. Gurfr vagreiragvbaf jrer pbagenel gb jung V rkcrpgrq ng gur fgneg. Gura V jnf guvaxvat nobhg ubj gur erqhpr gur pregnva qrngu rapbhagref naq gung ybj evfx rapbhagref jrer tbbq. Guvf vagreiragvba frrzf gb or gb vapernfr gur punapr bs gur fznyyrfg evfx rapbhagref (rira vg guvf vapernfrf uvtu evfx rapbhagref nf jryy) naq ubcr sbe gur orfg.

Viktor Rehnberg 10 Oct 2022 11:26 UTC
4 points
0
in reply to: Sausage Vector Machine’s comment on: Why I think there’s a one-in-six chance of an imminent global nuclear war
Lol. Somehow $1 : 100000000$ made it more clear that it was meant as a hyperbole than $1 : \infty$ did.

Viktor Rehnberg 20 Sep 2022 15:00 UTC
4 points
0
in reply to: interstice’s comment on: Prize idea: Transmit MIRI and Eliezer’s worldviews
Could someone who disagrees with the above statement help me by clarifying what the disagreement is?

Seeing as it has −7 on the agreement vote and that makes me think it should be obvious but it isn’t to me.

Viktor Rehnberg 1 May 2022 13:40 UTC
4 points
on: How confident are we that there are no Extremely Obvious Aliens?
I have not seen anyone do something like this but it sounds like something Anders Sandberg (FHI) would do. If you want a lead or want to find someone that might be interested in researching it, he might be it.

Viktor Rehnberg 12 Apr 2021 9:11 UTC
4 points
in reply to: Remmelt’s comment on: Some blindspots in rationality and effective altruism
Thanks, I realised that I provided zero context for the figure. I added some.
- is x like the input data?
- could y correspond to something like the supervised (continuous) labels of a neural network, which inputs are matched too?
Yes. The example is about estimating y given x where x is assumed to be known.
- does epistemic uncertainty here refer to that inputs for x could be much different from the current training dataset if sampled again (where new samples could turn out be outside of the current distribution)?
Not quite, we are still thinking of uncertainty only as applied to y. Epistemic uncertainty here refers to regions where the knowledge and data is insufficient to give a good estimate y given x from these regions.
To compare it with your dice example, consider x to be some quality of the die such that you think dies with similar x will give similar rolls y. Then aleatoric uncertainty is high for dies where you are uncertain for values of new rolls even after having rolled several similar dies and rolling more similar dies will not help. While epistemic uncertainty is high for dies with qualities you haven’t seen enough of.

Viktor Rehnberg 22 Mar 2021 15:09 UTC
4 points
in reply to: Remmelt’s comment on: Some blindspots in rationality and effective altruism
This is my go-to figure when thinking about aleatoric vs epistemic uncertainty.
Source: Michel Kana through Olof Mogren
Edit: Explaining two types of uncertainty through a simple example. x are inputs and y are labels to a model to be trained.
Edit: In the context of the figure. The aleatoric uncertainty is high in the left cluster because the uncertainty of where a new data point will be is high and is not reduced by the number of training examples. The epistemic uncertainty is high in regions where there is insufficient data or knowledge to produce an accurate estimate of the output, this would go down with more training data in these regions.

Viktor Rehnberg 8 Apr 2024 7:25 UTC
3 points
0
in reply to: Malentropic Gizmo’s comment on: Should you refuse this bet in Technicolor Sleeping Beauty?
Good formulation. “Given it’s Monday” can have two different meanings:
- you learn that you will only be awoken on Monday, then it’s 50%
- you awake assign ¹⁄₃ probability to each instance and then make the update $P (T | M) = P (M | T) P (T) / P (M) = (1 / 2) (2 / 3) / (2 / 3) = 50 %$
So it turns out to 50 % for both but it wasn’t initially obvious to me that these two ways would have the same result.

Viktor Rehnberg 12 Oct 2022 16:42 UTC
3 points
0
in reply to: PeterMcCluskey’s comment on: Actually, All Nuclear Famine Papers are Bunk
Skimming the methodology it seems to be a definite improvement and does tackle the short-comings mentioned in the original post to some degree at least.

Viktor Rehnberg 5 Oct 2022 7:48 UTC
3 points
0
in reply to: Paul Tiplady’s comment on: All AGI safety questions welcome (especially basic ones) [~monthly thread]
There is no consensus about what constitutes a moral patient and I have seen nothing convincing to rule out that an AGI could be a moral patient.

However, when it comes to AGI some extreme measures are needed.

I’ll try with an analogy. Suppose that you traveled back in time to Berlin 1933. Hitler has yet to do anything significantly bad but you still expect his action to have some really bad consequences.

Now I guess that most wouldn’t feel terribly conflicted about removing Hitler’s right of privacy or even life to prevent Holocaust.

For a longtermist the risks we expect from AGI are order of magnitudes worse than the Holocaust.

Have these issues been discussed somewhere in the canon?

The closest thing of this being discussed that I can think of is when it comes to Suffering Risks from AGI. The most clear cut example (not necessarily probable) is if an AGI would spin up sub-processes that simulate humans that experience immense suffering. Might be that you find something if you search for that.

Viktor Rehnberg 7 Jul 2022 16:20 UTC
3 points
1
in reply to: Koen.Holtman’s comment on: Principles for Alignment/Agency Projects
I agree with 1 (but then it is called alignment forum, not the more general AI Safety forum). But I don’t see that 2 would do much good.

All narratives I can think of where 2 plays a significant part sounds like strawmen to me, perhaps you could help me?

Viktor Rehnberg

indexA

indexB

indexC

indexD

Performance analysis