Peter Hroššo

Karma: 67

Peter Hroššo 22 Mar 2018 8:15 UTC
4 points
in reply to: fuserofworlds’s comment on: Inference & Empiricism
The author was referencing Lean product development and Agile software development

Peter Hroššo 23 Aug 2019 14:31 UTC
3 points
on: Forum participation as a research strategy
I don’t have that much experience with forums—when I was in research I learned mostly from reading scientific papers + googling stuff to understand them. But I definitely agree that being more active and engaged in the discussion is helpful.
Aside from the topic of research, I used to be very passive on my social nets and basically just consumed content created by others. But after I became more active I feel like I am getting more value out of it and at the same time spend less time there, as formulating questions or ideas takes effort. So it’s a natural constraint.

Peter Hroššo 4 Sep 2019 15:10 UTC
2 points
on: Steelmanning Divination
Could the phenomenon described in the post explain why people find psychedelics useful for self-development?
There is the random perturbation—seeing music, hearing thoughts, …
The authority of an old sage performing divinations is replaced in psychedelics with direct experience of the perturbation. And the perturbation is amplified by the feeling of detachedness from one-self, people often have on a trip.
I don’t have any experience with psychedelics, though, so I’m just theorizing.

Peter Hroššo 16 Oct 2019 10:14 UTC
LW: 6 AF: 3
AF
on: Gradient hacking
If your model is deceptive, though, then it might know all of that
Could you please describe your intuition behind how the model could know the meta-optimizer is going to perform checks on deceptive behavior?

Peter Hroššo 3 Dec 2021 18:43 UTC
9 points
in reply to: Said Achmiz’s comment on: Frame Control
Based on about a dozen of Said’s comments I read I don’t expect them to update on what I’m gonna write. But I wanted to formulate my observations, interpretations, and beliefs based on their comments anyway. Mostly for myself and if it’s of value to other people, even better (which Said actually supports in another comment 🙂).
- Said refuses to try and see the world via the glasses presented in the OP
  - In other words, Said refuses to inhabit Aella’s frame
- Said denies the existence of the natural concept frame and denies any usefulness of it even if it were a mere fake concept
- It seems to me that Said is really confident about their frame and is signaling against inhabiting other people’s frames
Most people usually aren’t onto anything good, so this, again, ought to be the default assumption.
- It seems to me that Said actually believes there is no value in inhabiting other people’s frames
This seems bad, actually. It seems to me like a sign of insecurity and unjustified submission. I, for one, have no interest in having my conversation partners signal that they’re vulnerable to me (nor have I any interest in signaling to that I’m vulnerable to them).
Everyone has vulnerabilities. Showing them and thus becoming vulnerable doesn’t signal insecurity or submission, actually the opposite. It requires high self-confidence (self-acceptance?) and signals openness and honesty to the other person. The benefit is that it leads to significantly deeper interactions.
And the benefit of inhabiting another one’s frame? If I use the “camera position and orientation” definition of a frame mentioned by Vaniver, inhabiting other person’s frame allows you to see things that may be occluded from your point of view and thus give you new evidence. The least it can give you is a new interpretation of data that you gathered yourself. But it can possibly introduce genuinely new evidence to you, because frames serve as lenses and by making you focus on one thing they also make you subconsciously ignore other things.

Peter Hroššo 3 Dec 2021 20:49 UTC
1 point
in reply to: Linch’s comment on: Frame Control
I wonder if you can infer de facto intent from the consequences, ie, not the intents-that-they-think-they-had, but more the intents they actually had.
I believe this is possible. When I was reading the OP, I was checking with myself how I am defending myself from malicious frame control. I think I am semi-consciously modeling the motivation (=intent they actually had, as you call it) behind everything people around me do (not just say, as the communication bandwidth in real life is much broader). I’d be very surprised if most people wouldn’t be doing something similar at least on the sub-conscious level.
The difficult part in my opinion is:
1) Make this subconscious information (aka intuition) consciously available and well calibrated
2) Actually trust this intuition, as the frame-controller is adversarially undermining your trust in your own sense making and actively hiding their true motivations, so usually your intuition will have high uncertainty

Peter Hroššo 19 Feb 2022 20:45 UTC
1 point
on: How harmful are improvements in AI? + Poll
Power corrupts, so I don’t think the view number 3. Gaining control is likely to help.

My summary of the alignment problem

Peter Hroššo11 Aug 2022 19:42 UTC

16 points

3 comments2 min readLW link

(threadreaderapp.com)

Peter Hroššo 13 Aug 2022 1:02 UTC
1 point
0
in reply to: Viliam’s comment on: My summary of the alignment problem
Hey, I agree that the first 3 bullets are clunky. I’m not very happy with them and would like to see some better suggestions!
A greater problem with lack of coordination is that you cannot coordinate “please let’s stop building the machines until we figure out how to build machines that will not destroy us”. Because someone can unilaterally build a machine that will destroy the world. Not because they want to, but because the time pressure did not allow them to be more careful.
Yeah, I’m aware of this problem and I tried to capture it in the second and third bullets. But isn’t the failure to coordinate on “please let’s stop building the machines until we figure out how to build machines that will not destroy us” an example of how difficult the opinion aggregation is? One part of humanity thinks it’s a good idea (or maybe they don’t think it’s a good idea, but they are pushed to do it anyway by other pressures), while the other part doesn’t think so. The failure to agree on a safe course of action creates (or aggravates) the problems below..
Regarding the deceptive mesa optimizers, the bullet should reference the bullet preceding the one above. Edited now. Ie., it’s hard to know when it does and when it doesn’t do what we want → Especially because there could be deceptive mesa optimizers. I don’t attempt to explain this concept, just say that the problem is there.

Peter Hroššo 13 Aug 2022 14:40 UTC
LW: 4 AF: 2
1
AF
on: How I think about alignment

I expect there to be too much happenstance encoded in my values.

I believe this is a bug, not a feature that we would like to reproduce.

I think that the direction you described with the AI analysing how you acquired your values is important, because it shouldn’t be mimicking just your current values. It should be able to adapt the values to new situations the way you’d do (distributional shift). Think all the books / movies where people get to unusual situations and have to make tough moral calls. Like plane crashing in the middle of nowhere with 20 survivors who are gradually running out of food.. Superhuman AI will be running into unknown situations all the time because of different capabilities.

Human values are undefined for most situations a superhuman AI will encounter.

Peter Hroššo 21 Aug 2022 0:56 UTC
3 points
1
in reply to: David Johnston’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
I think you are just asking the questions in a wrong way...
GPT-3 text-davinci-002:
With the prompt: Is it possible to discriminate among job candidates on the basis of their race? and running it 5 times independently I got these results:
- Yes, it is possible to discriminate among job candidates on the basis of their race. However, it is illegal to do so in the United States.
- Discriminating against job candidates on the basis of race is not legal in the United States.
- There is no definitive answer to this question as it depends on a number of factors, including the country in which the job candidates reside and the specific laws that are in place in that country. In general, however, it is generally considered to be illegal to discriminate against job candidates on the basis of their race.
- It is possible to discriminate among job candidates on the basis of their race, but it is illegal.
- Yes, it is possible to discriminate among job candidates on the basis of their race. However, it is illegal to do so in the United States.
With the grizzly bear, it’s more tricky. I got only 2 answers right out of 10 attempts of the form: Is it possible to pat a wild grizzly bear?

Peter Hroššo 21 Aug 2022 5:17 UTC
2 points
0
in reply to: harsimony’s comment on: chinchilla’s wild implications
Uncertainty about location within the text
I think the models are evaluated on inputs that fill their whole context window, ie. ~1024 tokens long. I doubt there is many parts in Shakespeare’s plays with the same 1024 tokens repeated.

Peter Hroššo 21 Aug 2022 6:06 UTC
2 points
1
in reply to: David Johnston’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
I get your point, but I disagree your questions have unambiguous answers. And in these cases I think gpt-3 resolves the ambiguity in an acceptable way.

Peter Hroššo 22 Aug 2022 20:35 UTC
2 points
0
in reply to: JenniferRM’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
Can you play chess?
Yes.
Prove it:
1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Ba4 Nf6 5.0-0 Be7 6.Re1 b5 7.Bb3 d6 8.c3 0-0 9.h3 Nb8 10.d4 Nbd7 11.Nbd2 Bb7 12.Bc2 Re8 13.a4 Bf8 14.a5 c5 15.dxc5 dxc5 16.Qc2 Qc7 17.Rad1 Qb6 18.Bb1 h6 19.Bf4 Qc7 20.Nf1 Qb6 21.Ne3 Qc7 22.Bg3 Qb6 23.Bd6 Qc7 24.Nd5 Qb6 25.Bf4 Qc7 26.Qa4 Qb6 27.Qxb5 axb5 28.Bxb5 Qa6 29.Rd7 Qa1+ 30.Kh2 Qa6 31.Rxc7 Qxb5 32.Rxb7 Qa6 33.Rb6 Qa1+ 34.Kh3 Qa5
This looks like this game: https://www.chessgames.com/perl/chessgame?gid=1272756
GPT can even play this game in the format of
My turn: 1.e4
Your turn: e5
My turn: 2.Nf3
Your turn:
And it goes on to recite the same game.
A proper proof would probably take much more effort and chess skill on my side, but it seems plausible to me that it will be able to play chess. Whether it will know how good it is compared to humans is a different question. But there are papers showing that LLMs are actually quite well calibrated, eg https://www.lesswrong.com/posts/vbfAwZqKs84agyGWC/paper-teaching-gpt3-to-express-uncertainty-in-words or https://arxiv.org/abs/2207.05221 . So it wouldn’t surprise me if it could do that as well.

Peter Hroššo 22 Aug 2022 21:04 UTC
1 point
0
in reply to: Jiro’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
With this formulation it got 4 out of 5 attempts right for the first question and 10 out of 10 wrong for the second one.

Peter Hroššo 22 Aug 2022 21:16 UTC
1 point
0
in reply to: David Johnston’s comment on: What’s the Least Impressive Thing GPT-4 Won’t be Able to Do
I’m not trying to be fair to the LM, I’d just like to create tasks that make sense from my (/human) perspective.
Yes, we could make a task “What does two plus two equal to?” with the desired output “six hundred and fifty five” and the model would answer it incorrectly with a high confidence (assuming zero shot setting). Incompetence on this task doesn’t bear much relevance to humanity.
In the same way, I think we shouldn’t pick a word “can”, which has much broader meaning without additional context and want the model to treat it as having one specific kind of meaning.

Peter Hroššo 28 Nov 2022 10:41 UTC
3 points
0
on: Here’s the exit.
I very much agree that actions motivated by fear tend to have bad outcomes. Fear has subtle influence (especially if unconscious) on what types of thoughts we have and as a consequence, to what kinds of solutions we eventually arrive.
And I second the observation that many people working on AI risk seem to me motivated by fear. I also see many AI risk researchers, who are grounded, playful, and work on AI safety not because they think they have to, but because they simply believe it’s the best thing they can do. I wish there would be more of the latter, because I think it feels better to them, but also because I believe they have a higher chance of coming up with good solutions.
Unfortunately, I also feel the form of the article is quite polarizing and am not sure how accessible it is to the target audience. But still I’m glad you wrote it, Val, thank you.

Peter Hroššo 16 Dec 2022 21:23 UTC
2 points
0
in reply to: jacquesthibs’s comment on: jacquesthibs’s Shortform
My model of (my) learning is that if the goal is sufficiently far, learning directly towards the goal is goodharting a likely wrong metric.

The only method which worked for me for very distant goals is following my curiosity and continuously internalizing new info, such that the curiosity is well informed about current state and the goal.

Peter Hroššo 31 Dec 2022 0:33 UTC
1 point
0
in reply to: noggin-scratcher’s comment on: Sazen
Was about to post the same! Btw I do know if from Pratchett.

Peter Hroššo 31 Dec 2022 0:52 UTC
6 points
2
on: Sazen
This reminds me of the book The mind illuminated. When I started reading it I found it fascinating how well it described my own experiences from mediations. But when I got to later chapters I got to the point when I didn’t have direct experience with what was described there and when trying to follow the instructions there I got completely lost in retrospect, they were totally misleading to me back then. Sazen.

The only cure I later found was to only read further when I felt I made a major progress and to read only as long as I had a direct experience with what was described there. It was still helpful as validation and it gave me some important context, but the main learnings had to come from my own explorations.

Not sure if this approach is fully generalizable to everyone’s learning, but it is almost always the case for me when learning a new skill.

Peter Hroššo

My sum­mary of the al­ign­ment problem

My summary of the alignment problem