ZT5

Karma: 153

Is it time to start thinking about what AI Friendliness means?

ZT511 Apr 2022 9:32 UTC

18 points

6 comments3 min readLW link

ZT5 1 Jul 2022 23:13 UTC
14 points
4
on: Who is this MSRayne person anyway?
To offer a data point, my reaction to your post was not “this person is weird and I should make fun of them”, it’s more of “this person is interesting and it’s good that more people are being open about their ‘weirdness’/non-typicalness”.
Aside from this, I’m not sure I have any useful advice to give. I never quite figured out how to use my intelligence towards being competent at dealing with reality.

ZT5 30 Apr 2022 21:22 UTC
13 points
in reply to: Valentine’s comment on: The Game of Masks
That’s an interesting observation! I’ve had something like this experience when teaching programming, going from trying to explain my approach to “oh I see, you have a different approach and it makes sense too”. Or just getting some partial understanding of what mental algorithm the student is executing and in with ways it fails to match with reality.
what they’re doing makes sense on the inside
I am wary of trying to understanding people too hard. A lot of things people say is a network running in reverse, a bullshit generator for whatever conclusion they already decided on, whatever outcome that’s socially convenient to them. Sure, it makes sense to them on the inside—but soon (if it’s more convenient to them) they’ll believe a slightly different, or a very different thing, without ever noticing they changed their mind.
I suppose the true-understanding here would be to notice the parts of their belief systems are more “real”/invariant, the parts are that are unreal/contradictory, and the invisible presence in the background that is pressuring the belief system into whatever shape it is at the moment.

ZT5 2 Feb 2015 14:05 UTC
13 points
on: Open Thread, Feb. 2 - Feb 8, 2015
How would you respond if I said I’m a rationalist, however I don’t feel a strong motivation to make the world a better place?

To be clear, I do recognize making the world a better place a good thing, I just don’t feel much intrinsic motivation to actually do it.

I guess in part it’s because I expect genuinely trying to improve things (rather than making a token effort) to be a rather difficult and thankless task.

Also, as far as I can tell, my psychological makeup is such that feeling, thinking or being told that I’m “obligated” to do something actually decreases my motivation. So the idea that “I’m supposed do that because it’s the ethical thing to do” doesn’t work for me either.

I do like the idea of making the world a better place as long as I can do that while doing something that inspires me or that I feel good about doing. Part of the reason, I think, is that I don’t see myself being able to do something I really don’t enjoy for long enough that it produces meaningful results. So in order for it work, it pretty much has to be something I actually like doing.

In the end, I estimate that I’m more likely to accomplish things with social benefit if I focus on my own needs and wait until I feel inspired to do something for others (or until there’s an overlap between meeting my needs and doing things for others), rather than trying to force an intention to do things for others (and then feel I’m not being honest with myself and that I don’t actually have that intention).

I don’t know how to feel about that.

ZT5 9 Apr 2022 13:47 UTC
11 points
on: It’s time for EA leadership to pull the fast-takeoff fire alarm.
I have some concern that AI risk advocacy might lead someone to believe the “AI is potentially civilization-altering and fast-takeoff is a real possilibity within 10 years” part but not the “alignment is really, really, really hard” part.
I imagine you can see how that might lead to undesirable outcomes.

ZT5 9 Apr 2022 1:18 UTC
11 points
on: A method of writing content easily with little anxiety
Interesting!
Since about 2-3 weeks back, this is how I write.
Based on some kind of writer-instinct. Markov chain completion. “Babbling”. Going with the flow. I simply output words, until I reach the point where it feels right to stop.
Corrections and edits are the same ways. Look over the text and go with the feeling. Add more text where it feels incomplete, where the transitions feel abrupt. Sometimes this type of corrections redirect the text on a new trajectory, making the old version obsolete. Sometimes not.
Having never written fiction before, I wrote an 8k word story in 3-4 days (and abandonded, because it was kinda cringy). Still, I was really happy with the attempt.

ZT5 12 Jun 2022 0:00 UTC
9 points
in reply to: Jeff Rose’s comment on: Poorly-Aimed Death Rays
I think the point of this post is “a powerful enough optimization process kills you (and everyone else) anyway”.
As soon as you give it a command the AI has “volition” in the sense that it is optimizing some output that affects the world.

ZT5 30 Apr 2022 20:55 UTC
9 points
on: We transhumanists want immortality… But is it really possible?
This feels like a… strange framing to me.
The thing that appeals to me about “immortality” is not having to deal with the problem of my lifespan being limited.
Actual infinity/eternity is really long time. I mean, I could live for a million (or billion, or trillion) years, and it still would be an small fraction of eternity. An infinitesimal fraction, in fact. Epsilon.
Do I choose between being forced to exist forever, or to die after less than 100 years of existence? Neither. I’d like to have the option to keep living for as long as I want. I can see the scenario of even people living in utopia eventually growing tired/bored/unhappy with their existence. I can see the scenario of this never happening, a person choosing to truly live forever (one day/month/year at a time. not by pre-commiting to living forever).

ZT5 29 Apr 2022 7:22 UTC
8 points
on: A Parable Of Explainability
I was going to make the point that I can look at the Wikipedia article for Higgs boson and gain a somewhat-decent understanding of it. Having tested it, thas hypothesis was disproven: after 1 minute and 30 seconds reading Wikipedia I conclude that even a cursory understanding of the Higgs Boson requires knowledge I do not have (it gives some particles mass. I get that part). Kudos, it was a good example.

ZT5 12 Apr 2022 1:32 UTC
8 points
on: Godshatter Versus Legibility: A Fundamentally Different Approach To AI Alignment
Ok, this is alignment 101. I hate to be so blunt, but you are making a very obvious error, and I’d rather point it out.
A paperclip-maximizer, or other AI with some simple maximization function, is not going to care if it’s born in a nice world or a not-nice world. It’s still going to want to maximize paperclips, and turns us all into paperclips, if it can get away with it.
You seem to be anthropomorphising AI way too much. An AI, by default, does not behave like a human child. An AI, by default, does not have mirror neurons or care about us at all. An AI, by default, has a fixed utility function and is not interesting in “learning” new values based on observing our behavior.
Nor is an AI some kind of slave. An AI acts according to its values and utility function. An unaligned artificial superintelligence does that in a way that is detrimental to us and very likely destroys human civization, or worse. An aligned artificial superintelligence does that is a way is beneficial to us, leading to some kind of human utopia and civilizational immortality.
The idea that AI has its own preferences and values and that we would need to “cooperate” with it and “convince” it to act in our interests is ridiculous to begin with. Why would we create a superintellingence that would want things that don’t perfectly align with our interests? Why would we create a superintelligence that would want to harm us? Why would we create a superintelligence that would want things that are, in any way, different from what we want?
The safe thing to do if a civilization is aligned to its own values is not to leave a nice message for any AGI that might happen to come into existence, hoping it might choose to cooperate with us (it won’t. or it will, until it betrays us, because it never cared about us to begin with).
The safe thing to do is not to create any AGIs at all until we are very certain we can do it safely, in a way that is perfectly aligned with human values.

ZT5 8 Jun 2022 11:52 UTC
7 points
0
in reply to: svemirski’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Also if you give edit access to the AI’s mind then a sufficiently smart AI whose reward is reducing other agent’s rewards will realise that its rewards are incompatible with the environment and modify its rewards to something compatible. To illustrate, Scott Aaronson wanted to chemically castrate himself because he was operating under the mistaken assumption that his desires were incompatible with the environment.
If the thing the AI cares about is in the environment (for example, maximizing the number of paperclips), the AI wouldn’t modify its reward signal because that would make its reward signal less aligned to the thing it actually cares about it.
If the thing the AI cares about is inside its mind (the reward signal itself), an AI that can self-modify would go one step further than you suggest and simply max out its reward signal, effectively wireheading itself. Then take over the world and kill all humans, to make sure it is never turned off and that its blissful state never ends.
I think the difference between “caring about stuff in the environment” and “caring about the reward signal itself” can be hard to grok, because humans do a bit of both in a way that sometimes results in a confusing mixture.
In humans, maintenance of final goals can be explained with a thought experiment. Suppose a man named “Gandhi” has a pill that, if he took it, would cause him to want to kill people. This Gandhi is currently a pacifist: one of his explicit final goals is to never kill anyone. Gandhi is likely to refuse to take the pill, because Gandhi knows that if in the future he wants to kill people, he is likely to actually kill people, and thus the goal of “not killing people” would not be satisfied.
However, in other cases, people seem happy to let their final values drift. Humans are complicated, and their goals can be inconsistent or unknown, even to themselves.
Suppose I go one step further: aliens offer you a pill that would turn you into a serial killer, but would make your constantly and euphorically happy for the rest of your life. Would you take the pill?
I think most humans would say no, even if their future self would be happy with the outcome, their current self wouldn’t be. Which demonstrates that humans do care about other things than their own “reward signal”.
In a way, a (properly-programmed) AI would be more “principled” than humans. It wouldn’t lie to itself just to make itself feel better. It wouldn’t change its values just to make itself feel better. If its final value is out in the environment, it would single-mindedly pursue that value, and not try and decieve itself into thinking it has already accomplished that value. (of course, the AI being “principled” is little consolation to us, if the its final values are to maximize paperclips, or any other set of human-unfriendly values).

ZT5 13 Apr 2022 7:54 UTC
6 points
on: How dath ilan coordinates around solving alignment
Thank you, this is a great compilation!

ZT5 4 Feb 2015 4:21 UTC
6 points
on: Open Thread, Feb. 2 - Feb 8, 2015
I’ve recently had a discussion about ethics here in this thread, and the conclusion I’ve arrived at is that a big reason for my lack of motivation is lack of social support.

I don’t know if this is the right place to post this, nor am I fully clear on what kind of response I am expecting. I guess I would like advice and emotional support with this issue.

I have basically been in shutdown mode for the past year because I’m not getting the kind of support I need, and I have my doubts I will ever get the kind of support I need.

I am in my mid-twenties, highly intelligent and have nonconformist opinions—I also have had personal difficulties and not lived a very happy life so far. I find myself unable to connect to other people when it comes to personal stuff because most people, even well-meaning ones, can’t understand what’s going on for me. That goes for mental health professionals as well. And unfortunately people, even mental health professionals, can be surprisingly mean if I point out that their well-meaning opinions or advice aren’t working for me—which usually ends up with them going into a death spiral of self-justification and/or hurt feelings.

I doubt therapy would work for me (based on previous experiences), because of the personal connection with the therapist not working, and the information and strategies offered to be of rather mediocre quality—mostly things I already know, things that are rather obvious, things that aren’t generally true and things that don’t apply to me. I doubt anything would actually work for me actually, except actually solving the underlying problem which is that I don’t have a support network, “tribe”, or whatever you want to call it. Or at least having a possible solution in sight.

It’s not so much about receiving support (although that’s a part of it) - I guess I would like to have something meaningful to do. Right now I have an at least partially altruistic mindset, and nothing to direct it at, because I have a hard time liking “people” at large. I would like to have a personal connection to a person or people who I actually feel that I like—because otherwise when I feel like being doing something positive either for specific people or for the world, I don’t even have an accessible example of someone who I would want to benefit from the results of that.

So yeah. I’m not sure what to do about this, because I’m not feeling very hopeful at the moment. I find that for whatever reason even things I imagine to would be very basic (like being understood by other people) are really hard to find in reality.

...I notice I am feeling confused about this, because the particular set of experiences I just described seems to be extremely non-typical among people in general, and I’m not expecting it to be

Edit: I don’t live in the US (I feel this is worth sharing because it affects the advice/options available).

ZT5 30 Apr 2022 10:11 UTC
5 points
in reply to: JBlack’s comment on: Saying no to the Appleman
Sure. I have an opposite problem, of ending the interaction with at least the minimum of politeness instead of anger or utter contempt. I typically manage it, though. (though I don’t actually have to deal with door-to-door salesmen ever, and phone sales extremely rarely. So it’s more people who approach me on the street to try and sell stuff or ask for money).
Nevertheless, this has been interesting to consider. While I’m not vulnerable to door-to-door sales, I recently noticed I may be vulnerable to less obvious variants of the same kind of manipulation. I feel this is an example of a general pattern in social interactions (or, say, work relations).
You feel that the true reason to say no is impolite/unacceptable to say.
You say something that sounds polite/reasonable instead, even if that is not your true reason (though it may be part of the reason).
You have now pseudo-committed to acting like it is your actual reason to say no. This has created an attack surface that can be exploited.

ZT5 4 Feb 2015 14:50 UTC
5 points
in reply to: ChristianKl’s comment on: Open Thread, Feb. 2 - Feb 8, 2015

Unfortunately I don’t think that a majority of academically trained psychologists have enough control over their own emotions to not get angry for bad reasons and go into self-justification.

In my experience that is accurate.

To be fair, as long as people stick to the psychologist-client script, and have more-or-less typical problems, they probably will get acceptable treatment.

However, pointing out that what the mental health person is doing isn’t working for me, for reasons that person doesn’t immediately recognize as valid isn’t sticking to the script. (and probably just being more intelligent than that person and having genuinely non-standard opinions isn’t sticking to the script either).

I don’t know whether your state reaches depression

That varies. To some extent, yes.

Do you do exercise?

I do regular exercise.

After exercise the second highest rated intervention on curetogether is to spend time with a pet. In the absence of human interaction, a dog can fill some of that niche. It can give you the feeling that there somebody who accepts you like you are.

That’s interesting. I think that might work for me, but I have I doubts about my ability to arrange for that to happen.

LW meetups are good

Don’t have one in my area (in responding to MathiasZaman’s comment, I edited the my original post to reflect that I’m not located in the US).

I would go if there was a meetup in my area.

Joining a sports team is also good.

Merely doing things along other people is typically not enough for me to form connections. And it doesn’t sounds interesting or fun enough to me to be worth doing for its own sake.

ZT5 4 Feb 2015 14:06 UTC
5 points
in reply to: MathiasZaman’s comment on: Open Thread, Feb. 2 - Feb 8, 2015

The obvious suggestion is going to (or starting) a local Less Wrong meetup. They’re a good way to meet people who can become your “tribe.”

I agree, and I would definitely visit one if there was one nearby (note: I don’t live in the US. Edited the original post to reflect that).

As for trying to start a meetup—intuitively I feel there are several reasons why that might be problematic. And I don’t know if there are enough (or any) rationalists in my area.

Another option (and one that worked very well for me for quite a while) was to move most of my in-group needs online. I don’t make a strong distinction between cyberspace and meatspace friendships, so this worked out pretty well.

Thank you for sharing.

The “bonobo rationalists” of tumblr have a skype group that has general conversation, if you need something to try out.

Thanks for the suggestion. I believe I have found the contact information for that.

What is important to keep in mind is that “having a tribe,” means that most of your interactions will (and maybe should?) be trivial and banal. You need to build a rapport with the people, so that your brain will more readily accept their praise and advice.

I don’t know if I need that. Maybe other people need to have trivial interactions with me to see me as in-group, I don’t know. My experience with that is that the trivial interactions are not a reliable indicator for the quality of non-trivial interactions.

...

I have a feeling that the implication here is that the way to form connections is to have a bunch of casual interactions (that’s my prior expectation for how many kinds of connections work, anyway).

Maybe that’s not really the implication, so I might be going on a tangent here… but I’d like to share this anyway.

Casual interactions work very poorly for me, and I have a feeling that that way to connect select against my particular mindset.

The problem with casual interactions (the way I see it) is that they put too much weight on similarity and agreement about relatively unimportant things.

It’s signalling “I’m similar to you in a lot of ways” as a proxy for signalling “I’m not crazy, trustworthy, have reasonable values, etc etc...”.

I think it’s kind of like using “academic achievement” as a proxy for “learning”, because trying to measure “learning” directly is too inconvenient. (I don’t know what the term here is, lost purpose?).

I’d rather have people directly tell me what they expect, so I can tell them whether I think I can live up to that—rather than having to signal that indirectly. The problem with signalling is that a lot of standard signals people are simply not true for me (and there’s dishonesty and self-deception involved in signalling because that obviously allows for stronger signals). For example, my thinking patterns are different from most people so I can’t use “yeah, I’ve had exactly the same experience” as a bonding thing.

And, generally speaking, I suspect there might not be enough self-awareness in people for me to be able to say “you know, we’re sitting here talking about X but I suspect you’re actually interested in Y. How about we talk about that directly?”. (or maybe there’s some kind of taboo against doing exactly that, I don’t know).

ZT5 2 Feb 2015 18:33 UTC
5 points
in reply to: [deleted]’s comment on: Open Thread, Feb. 2 - Feb 8, 2015
I understand that a rationalist can potentially have any kind of goals, not necessary altruistic ones.

The reason for bringing this up is that I want to see if this kind of topic can be discussed here on LW, at all. And me being an (aspiring) rationalist is very relevant information to this.

ZT5 27 Feb 2023 14:57 UTC
4 points
0
on: Bayesian Scenario: Snipers & Soldiers
I think this scenario is really valuable and gives you a nice intuitive feel for Bayesian updates.
I wish there were more “Bayesian toy models” like this.
(I have the “show P(sniper)” feature always enabled to “train” my neural network on this data, rather than trying to calculate this in my head)

ZT5 22 Feb 2023 15:46 UTC
4 points
5
in reply to: Bill Benzon’s comment on: The idea that ChatGPT is simply “predicting” the next word is, at best, misleading
My model is that LLMs use something like “intuition” rather than “rules” to predict text—even though intuitions can be expressed in terms of mathematical rules, just more fluid ones than we usually see “rules”.
My specific guess is that the gradient descent process that produced GPT has learned to identify high-level patterns/structures in texts (and specifically, stories), and uses them to guide its prediction.
So, perhaps, as it is predicting the next token, it has a “sense” of:
-that the text it is writing/predicting is a story
-what kind of story it is
-which part of the story it is in now
-perhaps how the story might end (is this a happy story or a sad story?)
This makes me think of top-down vs bottom-up processing. To some degree, the next token is predicted by the local structures (grammar, sentence structure, etc). To some degree, the next token is predicted by the global structures (the narrative of a story, the overall purpose/intent of the text). (there are also intermediate layers of organization, not just “local” and “global”). I imagine that GPT identifies both the local structures and the global structures (has neuron “clusters” that detect the kind of structures it is familiar with), and synergizes them into its probability outputs for next token prediction.

ZT5 2 Dec 2022 6:09 UTC
4 points
3
on: Biases are engines of cognition
Nice write-up! I’m glad someone brought up this idea.
Here’s my take on this:
The human mind is an engine of cognition. Evolutionarily speaking, the engine is optimized for producing correct motor-outputs. Whether its internal state is epistemically true or not does not matter (to evolution), expect insofar that affects present and future motor-outputs.
The engine of cognition is made of bias/heuristics/parts that reason in locally invalid ways. Validity is a property of the system as a whole: the local errors/delusions (partially) cancel out. Think something like SSC’s Apologist and Revolutionary: one system comes up with ideas (without checking if they are reasonable or possible), one criticises them (without checking if the criticism is fair). Both are “delusional” on their own, but the combined effect of both is something approaching sanity.
One can attempt to “weaponize” the bias to improve the speed/efficiency of cognition. However, this can cause dangerous cognitive instability, as many false beliefs are self-reinforcing: the more you believe it the harder it is to unbelieve it. A bias that reinforces itself. And once the cognitive engine has gone outside its stability envelope, there is no turning back: the person who fell prey to the bias is unlikely to change their mind until they crash hard into reality, and possibly not even then (think pyramid schemes, cults, the Jonestown massacre, etc).

ZT5

Is it time to start think­ing about what AI Friendli­ness means?

Is it time to start thinking about what AI Friendliness means?