This isn’t as strong of an argument as I once thought
What is the “this” you’re referring to? As far as I can tell I haven’t presented an argument.
weathersystems
Karma: 119
Do you have a link to the job posting?
Which personalities do we find intolerable?
I would say it feels like my brain’s built in values are mostly a big subgoal stomp, of mutually contradictory, inconsistent, and changeable values. [...]
it feels like my brain has this longing to find a small, principled, consistent set of terminal values that I could use to make decisions instead.
Here’s a slate star codex piece on our best guess on how our motivational system works: https://slatestarcodex.com/2018/02/07/guyenet-on-motivation/.It’s essentially just a bunch of small mostly independent modules all fighting for control of the body to act according to what they want.I don’t think there’s any way out of having “mutually contradictory, inconsistent, and changeable values.” We just gotta negotiate between these as best we can.
There are at least a couple problems with trying to come up with a “small, principled, consistent set of terminal values” you could use to make decisions.
You’re never gonna be able to do it in a way that covered all edge cases.
Even if you were able to come up with the “right” system, you wouldn’t actually be able to follow it. Because our actual motivational systems aren’t simple rule following systems. You’re gonna want what you want, even if your predetermined system says to do otherwise.
You don’t really get to decide what your terminal values are. I mean you can fudge it a bit, but you certainly don’t have complete control over them (and thank god).
Negotiating between competing values isn’t something you can smooth over with a few rules. Instead it requires some degree of finesse and moment to moment awareness.
Do you play any board games? In chess there are a lot of what we can call “values.” Better to keep your king safe, control the center, don’t double your pawns etc. But there’s no “small, principled, consistent set of” rules you can use to negotiate between these. It’s always gotta be felt out in each new situation.
And life is much messier and more complex than something like chess.
It sounds like both of you may have gone through the exercise of find terminal goals that work for you.
I “found terminal goals” in the sense that I tried to figure out what were the main things I wanted in life. I came up with some sort of list (which will probably change in the future). It’s a short list, but definitely not principled or consistent :D. Occasionally it does help to keep me focused on what matters to me. If I find myself spending a lot of time doing stuff that doesn’t go in one of those directions, I try to put myself more on track.
If you want I can try to figure out how I got there. But it seems like your more concerned with the deciding between competing values thing.
Inclusive genetic fitness seems like it may be a reasonable terminal goal to replace the subgoal stomp.
Ya definitely don’t do that. If you did that you’d just spend all your time donating sperm or something.
While these sound good, the rationale for why these are good goals is usually pretty hand wavy (or maybe I just don’t understand it).
At some point you just got to start with some values. You can’t “justify” all of your values. You got to start somewhere. And there is no “research” that could tell you what values to start with.
Luckily, you already have some core values.
The goals you should pursue are the ones that help you realize those values.but there are a ton of important questions where I don’t even know what the goal is
You seem to think that finding the “right” goals is just like learning any mundane fact about the world. People can’t tell you what to want in life like they can explain math to you. It’s just something you have to feel out for yourself.
Let me know if I’m miss-reading you.
Maybe a dumb question. What’s an EM researcher? Google search didn’t do me any good.
What do you think about the vulnerable world hypothesis? Bostrom defines the vulnerable world hypothesis as:
If technological development continues then a set of capabilities will at some point be attained that make the devastation of civilization extremely likely, unless civilization sufficiently exits the semian-archic default condition.
(There’s a good collection of links about the VWH on the EA forum). And he defines “semi-anarchic default condition” as having 3 features:
1. Limited capacity for preventive policing. States do not have sufficiently reliable means of real-time surveillance and interception to make it virtually impossible for any individual or small group within their territory to carry out illegal actions – particularly actions that are very strongly disfavored by > 99 per cent of the population.
2. Limited capacity for global governance. There is no reliable mechanism for solving global coordination problems and protecting global commons – particularly in high-stakes situations where vital national security interests are involved.
3. Diverse motivations. There is a wide and recognizably human distribution of motives represented by a large population of actors (at both the individual and state level) – in particular, there are many actors motivated, to a substantial degree, by perceived self-interest (e.g. money, power, status, comfort and convenience) and there are some actors (‘the apocalyptic residual’) who would act in ways that destroy civilization even at high cost to themselves.
To me, the idea that we’re in a vulnerable world is the strongest challenge to the value of technological progress. If we are in a vulnerable world, the time we have left before civilizational devastation is partly determined by our rate of “progress.”
Bostrom doesn’t give us his probability estimate that the hypothesis true. But to me it seems quite likely that at some point we’ll invent the technology that will screw us over (if we haven’t already). AI and engineered pandemics are the scariest potential examples for me.
Do you disagree with me about the probability of us being in a vulnerable world? Do think we can somehow avoid discovering the civilization destroying tech while only finding the beneficial stuff?Or do you think we are in a vulnerable world, but that we can exit the “semi-anarchic default condition?” Bostrom’s suggestions (like having complete surveillance combined with a police state) for exiting the semi-anarchic default condition seem quite terrifying.
If you’ve written or spoken about this somewhere else, feel free to just point me there.
You may be interested in this 80000 hours podcast: Nova DasSarma on why information security may be critical to the safe development of AI systems
Do you have anything else you remember about the statement? Where you heard it, when you heard it etc.
I’m not so sure I get your meaning. Is your knowledge of the taste of salt based on communication?
Usually people make precisely the opposite claim. That no amount of communication can teach you what something subjectively feels like if you haven’t had the experience yourself.
I do find it difficult to describe “subjective experience” to people who don’t quickly get the idea. This is better than anything I could write: https://plato.stanford.edu/entries/qualia/.
The quotes above are not the complete conversation. In the section of the discussion about AGI, Blake says:
Blake: Because the set of all possible tasks will include some really bizarre stuff that we certainly don’t need our AI systems to do. And in that case, we can ask, “Well, might there be a system that is good at all the sorts of tasks that we might want it to do?” Here, we don’t have a mathematical proof, but again, I suspect Yann’s intuition is similar to mine, which is that you could have systems that are good at a remarkably wide range of things, but it’s not going to cover everything you could possibly hope to do with AI or want to do with AI.
Blake: At some point, you’re going to have to decide where your system is actually going to place its bets as it were. And that can be as general as say a human being. So we could, of course, obviously humans are a proof of concept that way. We know that an intelligence with a level of generality equivalent to humans is possible and maybe it’s even possible to have an intelligence that is even more general than humans to some extent. I wouldn’t discount it as a possibility, but I don’t think you’re ever going to have something that can truly do anything you want, whether it be protein folding, predictions, managing traffic, manufacturing new materials, and also having a conversation with you about your grand’s latest visit that can’t be… There is going to be no system that does all of that for you.
I don’t think he’s making the mistake you’re pointing to. Looks like he’s willing to allow for AI with at least as much generality as humans.
And he doesn’t seem too committed to one definition of generality. Instead he talks about different types/levels of generality.
Why would self-awareness be an indication of sentience?
By sentience, do you mean having subjective experience? (That’s how I read you)
I just don’t see any necessary connection at all between self-awareness and subjective experience. Sometimes they go together, but I see no reason why they couldn’t come apart.
Gary Musk decided
https://github.com/search for when stackoverflow fails me. Sometimes when I’m trying to figure out how to use some library with not great documentation, there are good examples in other people’s code that aren’t yet on stackoverflow.
product reviews on reddit (google search something like “light phone review site:reddit.com″)
https://en.wikipedia.org/wiki/Berkson%27s_paradox
I also liked this numberphile video about it: Link
Ah. Ya that makes sense. It sounds like it’s not so much about what to do in the moment of panic as what to focus on throughout your day-to-day life. Let yourself be interested in and pay attention to things other than that you feel bad all the time. Don’t let your pain be your main/only focus.
I read it as an analogy to a programming stack trace, but with motivations. Often times you’re motivated to do A in order to get B in order to get C, where one thing is desired only as a means to get something else. Presumably these chains of desire bottom out in some terminal desires, things that are desired for their own sake, not because of some other thing it gets you.
So one example could be, “I want to get a job, in order to get money, in order to be able to feed myself.”
I’m not sure if that’s what they meant. I’m often kind of skeptical of that sort of psychologizing though. It’s not that it can’t be done, but that our reasons for having motivations are often invisible to ourselves. My guess is that when people try to explain their own actions/motivations in this way, they’re largely just making up a plausible story.
Thanks for writing this. As someone who went through something very similar, I largely agree with what you wrote here.
To make the “accept the panic” bit a more concrete: following someone’s advice, when I’d start to panic, I’d sit down and imagine I was strapped to the chair. I’d imagine my feelings were a giant wave washing over me, but that I couldn’t avoid them, because I was strapped to the chair. The wave wouldn’t kill me though, just feel uncomfortable. I’d repeat that in my head “this is uncomfortable but not dangerous. this is uncomfortable but not dangerous...” Turns out that if you don’t try to avoid the bad feelings, they don’t last as long. My understanding is that by just sitting and taking it without flinching, you’re teaching your brain that panic is not something to be feared which reduces their intensity and frequency.
Before doing that I felt terrible for about an hour. With that technique it was reduced to about 15 minutes, then I quickly (in a week or two) stopped having panic attacks.
I’m not sure I understand how “Three, distract yourself.” fits with accepting panic though. I know for me, distracting myself was a way of not accepting. Of trying not to feel bad.
There are a few things that sound similar to what you’re talking about. The first is the process of writing an RFC: https://github.com/inasafe/inasafe/wiki/How-to-write-an-RFC. Also wikipedia must need to do many of the things you describe, so looking into how they reach consensus may be interesting for you. Also, there are attempts to have more of a direct democracy style governance in the US, and they have certain procedures that you may want to look into: https://www.newyorker.com/news/the-future-of-democracy/politics-without-politicians
I do like the idea of templates for certain types of discussion. That’s why I wrote this: https://www.lesswrong.com/posts/xE7F4b34pfTMThYMX/what-questions-should-we-ask-ourselves-when-trying-to.
So, depending on the topic and which appropriate template is chosen the chances of success are ‘almost’ guaranteed because the underlining logic is agreed upon and already proven.
It’s not easy for me to understand how strong of a claim you’re making here, because you say “depending on the topic” and “almost.” It still feels too strong to me. I’d say most of the time, at best, templates for discussion would just be helpful. Especially if people have different values and beliefs about the world, disagreements are very difficult to settle.
I suppose questions in mathematics or something where you can prove an answer is correct may be a type of exception. Check out the polymath project if you haven’t seen it already for an example of people collaborating on trying to solve (math) problems.
I have a lot of similar ideas to the ones you’ve presented in this post, so if you’d like to discuss these things anytime, feel free to send me a dm.
Heh, I got the same feeling from the Dutch people I met. My ex wife once did a corporate training thing where they were learning about the power of “yes and” in improve and in working with others. She and one other European person (from Switzerland maybe?) were both kinda upset about it and decided to turn their improve into a “no but” version.
Ya I definitely took agreeableness == good as just an obvious fact until that relationship.