Noosphere89

Karma: 170

Noosphere89 27 Sep 2022 12:55 UTC
1 point
0 ∶ 0
in reply to: jacob_cannell’s comment on: How should DeepMind’s Chinchilla revise our AI forecasts?

Also there is now mounting evidence that these LLMs trained on internet scale data are memorizing all kinds of test sets for many downstream tasks, a problem which only gets worse as you try to feed them ever more training data.

Not really? If we assume that they just memorize data without having intelligence, then their memory requirements would scale as N parameters, when instead we see a smaller constant for compression, which essentially requires actual intelligence rather than simply memorizing all that data.

Noosphere89 26 Sep 2022 12:22 UTC
1 point
0 ∶ 0
in reply to: janus’s comment on: Simulators
I want to talk about why automation is likely more dangerous and more useful than cyborgization, and the reason is Amdahl’s law.

In other words, the slowest process controls the outcome, and at very high levels, the human is likely to be the biggest bottleneck, since we aren’t special here.

Furthermore, I think that most interesting problems are in the NP complexity class assuming no deceptive alignment has happened. If that’s true, then goodhart that is non-adversarial is not a severe problem even with extreme capabilities, because while getting a solution might be super hard, it’s likely but not proven that p doesn’t equal np, and if that’s true than you can verify whether the solution actually works once you have it easily, even if coming up with solutions are harder.

Noosphere89 25 Sep 2022 22:15 UTC
4 points
0 ∶ 0
on: There’s One Thing Futurists are Never Wrong About
Weakly downvoting due to clickbait title, as said by Sune.

However, this gets at an important crux: Is the technological completion conjecture correct? It basically says that for all technologies that aren’t physically impossible, that humanity or it’s descendants will complete all the technologies that can be invented.

The link to the technological completion conjecture is here:

https://nickbostrom.com/papers/future

Noosphere89 24 Sep 2022 20:06 UTC
11 points
2 ∶ 0
on: Two reasons we might be closer to solving alignment than it seems
Strong upvote for giving some outside perspective to the field, and this is an important point of why AI Alignment is likely to be tractable at all. It also means getting many more researchers and money, fast is important for AI Safety.

Noosphere89 24 Sep 2022 0:55 UTC
1 point
on: Some real examples of gradient hacking
I’d argue almost none right now. In many ways, total gradient hacking was mostly avoided by the fact that the 20th century broke the idea of improving the human species. In other words, we voluntarily forgoed capabilities increases to prevent what we see as bad things.

Had the 20th century gone differently, gradient hacking/genetic engineering would happen in the 21st century.

Noosphere89 23 Sep 2022 12:30 UTC
1 point
0 ∶ 0
in reply to: Conor Sullivan’s comment on: Understanding Conjecture: Notes from Connor Leahy interview
I’d argue that a major part of the problem really is long-term consequentialism, but I’d argue that this is inevitable at least partially as soon as 2 conditions are met by default:
1. Trade offs exist, and the value of something cannot be infinity nor arbitrarily large values.
2. Full knowledge of the values of something isn’t known to the agent.
It really doesn’t matter whether consequentialism or morality is actually true, just whether it’s more useful than other approaches (given that capabilities researchers are only focusing on how capable a model is.)

And for a lot of problems in the real world, this is pretty likely to occur.

For a link to a dentological AI idea, here it is:

https://www.lesswrong.com/posts/FSQ4RCJobu9pussjY/ideological-inference-engines-making-deontology

And for a myopic decision theory, LCDT:

https://www.lesswrong.com/posts/Y76durQHrfqwgwM5o/lcdt-a-myopic-decision-theory

Noosphere89 23 Sep 2022 0:42 UTC
1 point
0 ∶ 0
in reply to: Prometheus’s comment on: DeepMind alignment team opinions on AGI ruin arguments
If I had to say why I’d disagree with a pivotal act framing, PR would probably be the obvious risk, but another risk is that it’s easy to by default politicize this type of thing, and by default there are no guardrails.

It will be extremely tempting to change the world to fit your ideology and politics, and that gets very dangerous very quickly.

Now unfortunately this might really happen, we can be in the unhappy scenario where pivotal acts are required, but if it ever comes to that, they will need to be narrow pivotal acts, and the political system shouldn’t be changed unless it relates to AI Safety. Narrowness is a virtue, and one of the most important ones during pivotal acts.

Noosphere89 22 Sep 2022 13:21 UTC
2 points
0 ∶ 0
on: Is the game design/art maxim more generalizable to criticism/praise itself?
My answer for why this may happen, conditional on this actually being generalizable, is that people have surprisingly good intuitions that something is off or deserves praise, but when forced to explain, the brain makes up a plausible reason, instead of the true reason that something is wrong

[Question] Is the game design/art maxim more generalizable to criticism/praise itself?

Noosphere8922 Sep 2022 13:19 UTC

4 points

1 comment1 min readLW link

Noosphere89 22 Sep 2022 12:38 UTC
2 points
0 ∶ 0
in reply to: Martín Soto’s comment on: An issue with MacAskill’s Evidentialist’s Wager
I’m talking about the evidence from the Cosmic Microwave Background radiation, as well as a little bit of dark energy, which is converging on a picture of a universe that is flat, infinite and homogeneous, that is at the large scale there are no imbalances in the distribution of mass.

There’s already a lot of evidence from physics to bear on this question, which is what I’m talking about.

Noosphere89 21 Sep 2022 22:39 UTC
2 points
0 ∶ 1
on: An issue with MacAskill’s Evidentialist’s Wager
On acausal trade, an important point is that a lot of evidence is emerging that the universe is in fact infinitely large, which validates your thesis even more (and by extension, make’s MacAskill’s arguments nonsense.)

Noosphere89 20 Sep 2022 13:48 UTC
3 points
0 ∶ 1
in reply to: Chris Leong’s comment on: Alignment via prosocial brain algorithms
If I had to say where it fails, it fails to be robust to relative scale. One thing I notice in real life history is that it really requires relatively equivalent power levels, and without that, it goes wrong fast (human treatment of non-pet animals are a good example.)

Noosphere89 17 Sep 2022 21:45 UTC
1 point
0 ∶ 0
in reply to: Diffractor’s comment on: What’s the longest a sentient observer could survive in the Dark Era?
Crucially, these are all virtual beings once you start getting past the stelliferous era, so you can do a whole lot of tricks with subjective time.

Noosphere89 16 Sep 2022 19:25 UTC
1 point
0 ∶ 0
in reply to: GeneSmith’s comment on: Transhumanism, genetic engineering, and the biological basis of intelligence.
This suggests a technology with a poor level of readiness, both in it’s safety and versatility (the chemotherapy is really bad for any enhancement, and even restricted to medicine, I would be surprised if most diseases could be worse. Also, monogenic alterations are almost never that impactful in life, especially positively. So it has extremely limited versatility.)

Noosphere89 15 Sep 2022 21:01 UTC
3 points
0 ∶ 0
on: Transhumanism, genetic engineering, and the biological basis of intelligence.
i would like to ask how far the somatic genetic editing technology has gone, compared to gamete genetic editing like perhaps CRISPR.

Noosphere89 15 Sep 2022 13:22 UTC
1 point
0 ∶ 0
in reply to: Mo Nastri’s comment on: Why Do People Think Humans Are Stupid?
This is because in real life, speed and resources matter because they’re both finite. Unlike a Turing machine that can assume both arbitrarily high memory and time, we don’t have such things.

Noosphere89 14 Sep 2022 21:14 UTC
1 point
0 ∶ 0
in reply to: benjamincosman’s comment on: Why Do People Think Humans Are Stupid?
This. It would be a very surprising fact if they didn’t have Turing completeness, as it’s very easy to do so. Given arbitrary time and memory, any computer can do any task, but that’s not how real life works.

Noosphere89 14 Sep 2022 14:20 UTC
1 point
0 ∶ 0
in reply to: Mitchell_Porter’s comment on: In a lack of data, how should you weigh credences in theoretical physics’s Theories of Everything, or TOEs?
To explain to everyone, the reason the black hole information paradox has been solved is that 2 things happened:

First, the Page Curve was resolved as, yes black holes do preserve and shed information, thus we’ve resolved whether information is destroyed as Hawking and Preskill thought, or does unitarity prevail in black holes, preserving information. Now we know that information must be preserved. Incredibly they even got it to work in our own flat universe.

Second, they got rid of both string theory and the AdS-CFT Correspondence/Holographic universe assumption, removing 2 burdensome details from the solution. It’s inspired by string theory and the AdS/CFT Correspondence/Holographic universe theories, but they don’t rely on these assumptions to solve the paradox. We’ve finally managed to get the answer without relying on burdensome details.

Now how it happens is unfortunately unsolved, and here the best theories like wormholes are pretty speculative. But we don’t need the how in order to appreciate the answer to the original Black Hole Information Paradox.

Noosphere89 11 Sep 2022 22:40 UTC
1 point
0 ∶ 0
on: Which values are stable under ontology shifts?
This is a reason why the future could be very lovecraftian, and at least partially explains why (assuming moral realism is true) that moral actions seem so disgusting to our intuitions.

Noosphere89 10 Sep 2022 14:01 UTC
1 point
0 ∶ 0
in reply to: Hastings’s comment on: All AGI safety questions welcome (especially basic ones) [~monthly thread]
Yeah, humans are very misaligned with other animal species that are less powerful, and has driven a lot of species to extinction, so I don’t agree with the premise at all.

Noosphere89

[Question] Is the game de­sign/​art maxim more gen­er­al­iz­able to crit­i­cism/​praise it­self?

[Question] Is the game design/art maxim more generalizable to criticism/praise itself?