Yet Another “Rational Approach To Morality & Friendly AI Sequence”
Premise: There exists a community whose top-most goal is to maximally and fairly fulfill the goals of all of its members. They are approximately as rational as the 50th percentile of this community. They politely invite you to join. You are in no imminent danger.
Do you:
Join the community with the intent to wholeheartedly serve their goals
Join the community with the intent to be a net positive while serving your goals
Politely decline with the intent to trade with the community whenever beneficial
Politely decline with the intent to avoid the community
Join the community with the intent to only do what is in your best interest
Politely decline with the intent to ignore the community
Join the community with the intent to subvert it to your own interest
Enslave the community
Destroy the community
Ask for more information, please
Premise: The only rational answer given the current information is the last one.
What I’m attempting to eventually prove The hypothesis that I’m investigating is whether “Option 2 is the only long-term rational answer”. (Yes, this directly challenges several major current premises so my arguments are going to have to be totally clear. I am fully aware of the rather extensive Metaethics sequence and the vast majority of what it links to and will not intentionally assume any contradictory premises without clear statement and argument.)
It might be an interesting and useful exercise for the reader to stop and specify what information they would be looking next for before continuing. It would be nice if an ordered list could be developed in the comments.
Obvious Questions:
<Spoiler Alert>
What happens if I don’t join?
What do you believe that I would find most problematic about joining?
Can I leave the community and, if so, how and what happens then?
What are the definitions of maximal and fairly?
What are the most prominent subgoals?/What are the rules?
I feel like a jerk for saying this, but in the four days since you announced your intention to cut back on top-level posting for a while, this is the second top-level post you’ve made.
To be blunt, you are violating community norms by posting large quantities of material despite general disinterest or disapproval from other community members. Before making future posts, please try to calculate the expected value of their content for your readers. When in doubt, refrain from posting.
This discussion section is intended to have lower standards than the main section of the site, but even so those standards are much higher than those of almost any other internet discussion forum.
While I can’t speak for anyone else here, I would appreciate it if you would cease making top-level posts entirely until your karma rises above 0.
My last top level post currently has a karma of +15. The net of the rest of my comments (i.e. not including that post) over the same time period is +12.
That upvoted post was an apology for your substantive posting, not a substantive post itself.
I think the mean is a more meaningful statistic than the net, in this case.
Please take a look through the list of recently posted discussion topics, and note how often various authors post. At the moment, the only one approaching your frequency is draq, who is also heavily downvoted. While there are LW users who would be celebrated if they posted new material every day or two, you can mostly identify them by looking at the “Top Contributors” list on the bottom right of this page.
Also, I second Carl Shulman.
Seconding Carl changes your argument to this is the first substantive posting I’ve made in four days. Now it’s one in five days.
Other than not posting on a new given topic (while you have no active or live posts), what would you suggest? Personally, I would suggest a separate area (a playpen, if you will) where newbies are allowed to post and learn. You can’t truly learn anything of value just by watching. Insisting that a first attempt be done correctly on the first try under safe circumstances is counter-productive.
My last substantive post before this one was a total admitted disaster (make that my last two substantive posts). This one is hanging in there. Apparently I’ve learned something. If I, like draq, am being heavily downvoted—this post would be positive for anyone else.
Continuing the admitted disasters would have been an exercise of throwing good time after bad. I’m trying to wring all the knowledge (or functionality) I can out of each top-level post but they were done. Do you really want to say that regardless of what I’ve learned, you “would appreciate it if you would cease making top-level posts entirely” until I’ve paid for my previous errors through certain very limited activities?
I get why your original comment has such high karma. I always have been trying to calculate the expected value of their content for your readers. I argue that not giving credit for intent and some slack to newbies (especially those showing progress) is counter-productive to any goal of outreach.
mwaser, every person on this board (possibly excepting some transfers from Overcoming Bias) was once a newbie. I was once a newbie. My first toplevel was downvoted too. If you want to be safe, you lurk until you truly get what’s going on around you. People can in fact learn things that way.
Your apology post was full of applause lights: you admitted fault, claimed mitigating circumstance and benign intent, requested patience and gentleness, and offered a community service. And you got lots of applause.
However, it doesn’t say anywhere what it is that you claim to have suddenly understood (which spelling out, surely, would be essential to writing a newbie’s guide, wouldn’t it?). And apart from sprinkling the linguistic equivalent of tags in everything you’ve written since, it doesn’t look like your conduct has changed. You’re only attaching soothing particles with no meaning behind them, uttering statements of trivial agreement whenever convenient without following substantive advice, and claiming repeatedly to have learned some unspecified thing which makes you above disapproval.
My first toplevel (posted back when a toplevel on Main was worth the same karma as a toplevel in Discussion) was downvoted, and then I shut the hell up until I had something I was more sure of to say. If I had only waited until I was “done”, just as I’d been “done” with my first post, then it would not have gone well for me. Every time someone downvotes you, you are being told to recalibrate your caution. At this point, if I were you, I’d drop into lurk-mode for a few months.
here
All right, I’ll dissect that comment.
Okay: what mistaken assumptions about karma? What false beliefs did you have about karma, and how did they mislead your actions?
Okay: how does what underlying structure explain what apparently irrational behavior?
Okay: And those regularities and assumptions are...?
Okay: and I can find your list of these, and how you misunderstood them, where?
Which takes what form, please?
Such as?
And the process is? The expectations are? The terms mean? The structure/understanding is? What is the mystery you have unraveled here, please show the class.
Do tell. How does it make sense?
And the rules are...?
Ending the dissection here because comments can’t be arbitrarily long, and because it’s all the same. You throw around words labeling things you supposedly understand without ever describing those things. Over and over and over.
I expect that the error is in using opaque words one can’t unpack, hence with very vague intended meaning. Like guessing teacher’s password. One thus often remains protected from saying something that doesn’t have an interpretation under which it’s correct (even if intended interpretation is trivial or wrong).
Okay. So the comment is unclear and incomplete but not unwelcome with a +5 karma). Clearly, I need to slow down and expand, expand, expand. I’m willing to keep fighting with it and do that and learn. Where is an appropriate place to do so?
How about you answer any one of the questions I posed, right here? Take your pick. There’s plenty.
Umph! I am really not used to interacting with people mentally skilled enough that I have a really bad case of not knowing what I don’t know. I need to fix that.
Good one with the tags. I’m still recalibrating from it/working through all its implications.
I’m going off to work on one of the questions now.
All right, I’ll dissect that comment.
Okay: what mistaken assumptions about karma? What false beliefs did you have about karma, and how did they mislead your actions?
Okay: how does what underlying structure explain what apparently irrational behavior?
Okay: And those regularities and assumptions are...?
Okay: and I can find your list of these, and how you misunderstood them, where?
Which takes what form, please?
Such as?
And the process is? The expectations are? The terms mean? The structure/understanding is? What is the mystery you have unraveled here, please show the class.
Do tell. How does it make sense?
And the rules are...?
Ending the dissection here because comments can’t be arbitrarily long, and because it’s all the same. You throw around words labeling things you supposedly understand without ever describing those things. Over and over and over.
Could you point to an example please so I can try to evaluate how I implied something so thoroughly against my intent? I certainly don’t believe myself above disapproval.
Better to reply to the person you’re replying to, not yourself.
I never said I wanted to be safe. Please reread what I said.
Lurking until you truly get what’s going on around you is not the most effective (rational) way to learn. I can provide you a boatload of references supporting that if you wish.
Do you really want subpar newbies who will accept such irrationality just to maintain your peace and quiet? Particularly when a playground option is suggested? You could even get volunteers and never deal with the hassle.
Premise: It’s more rational for your goals, to just ignore a good rational proposal from an erring, annoying newbie who is trying to provide access to new resources for you (both newbies and structures for their care and feeding).
I just don’t get that.
Please taboo “rational”. It’s generally a good idea for this word.
Edit: Interestingly, exactly the same thing irked Alicorn, apparently independently.
I invite you to try to re-write this comment without the word “rationality” or its cousins.
Is effective a cousin? I suspect so since the easiest way to rewrite it would be to simply replace rational with effective. If not, assume that my rewrite simply does that. If so, can I get a motivation for the request? I’m not sure where you’re going or why “cousins” are disallowed.
By “cousins” I meant “rational”, “irrational”, “rationality”, “irrationality”, etcetera. “Effective” is not technically a cousin, but any form of search-and-replace would not be in keeping with the spirit of the exercise. Since you are confused, I will go into more detail, but I am nearing the last straw in trying to deal with you and won’t extend the courtesy again.
Do you mean: Lurking is slow compared to other strategies, lurking gets worse results for the newbie, lurking is worse for the rest of the community, lurking is inefficient, lurking fails altogether at achieving the objective, or something else?
This is meaningless until you explain the assertion you offer to support.
Nope. That doesn’t sound appealing at all. I would rather have zero subpar newbies, and instead of peace and quiet I want lively and productive signal with minimal noise. Also, “such irrationality” is presumptuous. Weren’t you going on about how LW is actually governed by structures and rules that you now understand that only look irrational? Where did that go?
Interestingly, your “option” is not so obviously and blindingly brilliant that I could only reject it as the solution to all my problems through sheer bloodymindedness. I don’t actually want LW to be attached to a rock-bottom-standards blog with a similar color scheme that purports to funnel newbies into the real deal. I think that would be bad. Yes, even if I never have to look directly at it without a pinhole camera and even if it’s minded by volunteers.
If you were demonstrating actual understanding of any relevant concepts… or if you were offering to personally do some work for the site instead of just throwing around vague plans for its expansion and calling it the provision of “access”… or if your proposal were actually good or “rational”… or, I’ll admit it, if you weren’t so annoying… then you’d be getting a better reception. This is, of course, a counterfactual.
I meant lurking is slow, lurking is inefficient, and a higher probability that it gets worse results for the newbie. I’m not sure which objective is being referred to in that clause. I retract those evaluations as flawed.
Yeah, I made the same mistake twice in a row. First, I didn’t get that I didn’t get it. Then I “got it” and figured out some obvious stuff—and didn’t even consider that there probably was even more below that which I still didn’t get and that I should start looking for (and was an ass about it to boot). What a concept—I don’t know what I don’t know.
The playground option was an idiot idea. I actually figured out that I don’t want to go there and stagnate before your comment. I’ve got this horrible mental image of me being that guy that whines in boot camp. Let me take a few days and come up with a good answer to one of your questions (once I’ve worked this through a bit more).
I’d say thank you and sorry for being an ass but I’m not sure of its appropriateness right now. (Yeah, that tag is still really messing with me ;-)
ETA: Still re-calibrating. Realizing I’m way too spoiled about obtaining positive feedback . . . . ;-) EDIT: Make that addicted to obtaining positive feedback and less accepting of negative feedback that I don’t immediately understand than I prefer to realize (and actually commenting on the first part seems to immediately recurse into hilarity)
You misunderstand. Your posts are not being downvoted specifically because people dislike you. Neither are draq’s. A downvote means, approximately, “I would like to see less of this.”
Yes. If you have actually learned something then your comments will reflect this and earn karma. You’ll be into the positives before you know it.
If this is so, you have been doing it very badly.
I’m sorry I have to be so blunt, but I have yet to see any indication that you have actually learned something.
OK. Got it.
I’ve already acknowledged that. But I’ve clearly been doing better with the “What I missed” explanation being +5 and this post only garnering −2 over two days as opposed to −6 in a few hours so I must have learned something.
I’ve also learned that we’ve reached the point where some people are tired enough of this thread that they will go through it karma down any comment by me and karma up any comment not agreeing with me. (I should go visit draq’s posts and disagree with him ;-)
Too ambiguous (something of premature abstraction). It’s not clear what most of the elements of this post refer to, so it’s not possible to have a clear discussion about them.
Too ambiguous. It’s not clear which elements aren’t clear to you, so it’s not possible to fix the problem.
Pretty much everything. To fix the problem, give an example.
Thank you very much.
Premise: Most developed nations are such a community although the goal is certainly not explicit.
Do you believe that premise is flawed?
I think that premise is very wrong. If “developed nations” is the model you had in mind while writing, I can understand why most commentors find this post confusing. I guessed you meant something like an internet community like LW. Attempting to abstract over these things seems problematic, as pointed out by Vladimir Nesov.
What does it mean to “join” a nation? To be “invited to join”? To choose whether to do so or not? In what sense does a nation have a top-level goal (explicit or otherwise)? In what sense is a nation rational or otherwise? How does a nation identify the goals of its members?
Acquiring citizenship is joining a nation. People who are not only allowed to acquire citizenship but encouraged to do so are “invited to join”. To choose whether to do so or not is to file the necessary papers and perform the necessary acts. I think that these answers should be obvious.
A nation has a top-most goal if all of its goals do not conflict with that goal. This is more specific than a top-level goal.
A nation is rational to the extent that its actions promote its goals. Did you really have to ask this?
How does a nation identify the goals of its members? My immediate reaction is the quip “Not very well”. A better answer is “that is what government is supposed to be for”. I have no interest and no intention to get into politics. The problem with my providing a specific example, particularly one that falls short in the rationality department from what was stated in the premise, is that people tend to latch on to the properties of the example in order to argue rather than considering the premise. Current “developed nations” are a very poor, imperfect, irrational echo of the model I had in mind but they are the closest existing (and therefore easily/clearly cited) example I could think of.
In fact, let me change my example to a theoretical nation where Eliezer has led a group of the best and brightest LessWrong individuals to create a new manmade-island-based nation with a unique new form of government. Would you join if invited?
And this is still too abstract. Depending on detail of the situation, either decision might be right. For example, I might like to remain where I am, thank you very much.
Worse, so far I’ve seen no motivation for the questions of this post, and what discussion happened around it was fueled by making equally unmotivated arbitrary implicit assumptions not following from the problem statement in the post. It’s the worst kind of confusion when people start talking about the topic as if understanding each other, when in fact the direction of their conversation is guided by any reasons but the content of the topic in question. Cargo cult conversation (or maybe small talk).
So I take it that you are heavily supporting the initial post’s “Premise: The only rational answer given the current information is the last one.”
Thank you. I didn’t clearly understand the need for the explicit inclusion of motivation before.
The reason I ask questions which you think have obvious answers is that I think the easily-stated obvious answers make large, blurry assumptions. For example:
What are the actions of a nation? The aggregate actions of the population? Those of the head of state? What about lower-level officials in government? Large companies based in the nation?
Ok, I should have started with a more basic question then. What does it mean for a nation to have any goal?
I agree that nations are not a great example. After all, acquiring citizenship usually means emigration, new rights of travel, change in economic circumstances and often loss of previous citizenship. All of these overwhelm any considerations about rationality of the new nation.
Ah. Now I see your point.
The actions of a nation are those which were caused by it’s governance structure like your actions are those which are caused by your brain. A fever or your stomach growling is not your action in the same sense that actions by lower-level officials and large companies are not the actions of a nation—particularly when those officials and companies are subsequently censured or there is some later attempt to rein them in. Actions of the duly recognized head of state acting in a national capacity are actions of the nation unless they are subsequently over-ruled by rest of the governance structure—which is pretty much the equivalent of your having an accident or making a mistake.
A nation has explicit goals when it declares those goals through it’s governance structure.
A nation has implicit goals when it’s governance structure appears to be acting in a fashion resembling rational behavior for having those goals and there is not an alternative explanation.
I proposed a specific source of ambiguity elsewhere in the thread.
How is “investigating is whether Option 2 is the only long-term rational answer” different from investigating which options are long-term rational answers? And why are you choosing to focus on the former, rather than on the latter?
As a heads up, that point has been addressed (from a slightly different angle) elsewhere in the thread. You might yet get it across better than I did, though.
ETA: Oh, I guess you’re objecting to the wording change which was made in response to the earlier comment. Carry on then.
“Option 2 is the only long-term rational answer” is a clear hypothesis. It is disproved if any of the other options is also a long-term rational answer. “Which options are long-term rational answers?” is a question, not a hypothesis.
Reread Einstein’s Arrogance
In other words, you’re still investigating the same things (possibly with different stopping criteria—e.g. you’d be done if you disproved your hypothesis), but you have substantial evidence in favor of your hypothesis already. Am I understanding you correctly?
I’m not sure the blog post you’re linking to is helpful, though. One could come up with your list of options without having done any prior investigation. In other words, unlike Einstein, it’s entirely plausible to be at the stage where you’re considering Option 2 without having evidence favoring Option 2 over the others. And even if you have 50% certainty in Option 2, that only implies 3-4 bits of evidence.
And I think the mistrust you see in the comments is due precisely to the absence of evidence from your post. Which is weakly evidence of absence. Granted, I don’t think your post is intended to present all your evidence, but seeing some of it first would help frame your discussion.
Upvote from me! Yes, you are understanding me correctly.
One could indeed come up with my list of options without having done any prior investigation. But would one share it with others? My pointing at that particular post is meant to be a signal that I grok that it is not rational to share it with others until I believe that I have strong evidence that it is a strong hypothesis and have pretty much run out of experiments that I can conduct by myself that could possibly disprove the hypothesis.
Skepticism is desired as long as it doesn’t interfere with the analysis of the hypothesis. If mistrust leads someone to walk away from a hypothesis that would be of great interest to them, if true, without fairly analyzing the hypothesis, that’s a problem.
Yes, I realize that I still am lacking some of the skills necessary to present and frame a discussion here. I should have presented an example as Vladimir pointed out. I’m under the impression that evidence isn’t necessarily appropriate at this point. If people would leap in to correct me if that is incorrect, it would be appreciated.
The question “which options are long-term rational answers?” corresponds immediately to the hypothesis “among the options are some long-term rational answers” and can be investigated in the same way.
Mind you, “long-term rational answer” is not well-defined; I guess you mean something influenced by ideas like Nash equilibrium and evolutionarily stable strategy. What is a “short-term rational answer”?
The post you link to is irrelevant to Misha’s reasonable question, except insofar as it contains discussion of hypotheses. If you really think that people here need to be educated as to what a hypothesis is, then a) it’d be better to link to a wikipedia definition and b) why are you bothering to post here?
Incorrect. Prove that one option is a long-term rational answer and you have proved the hypothesis “among the options are some long-term rational answers”. That is nowhere near completing answering the question “which options are long-term rational answers”
My hypothesis was much, much more limited than “among the options are some long-term rational answers”. It specified which of the options was a long-term rational answer. It further specified that all of the other options were not long-term rational answers. It is much, much easier to disprove my hypothesis than the broader hypothesis “among the options are some long-term rational answers” which gives it correspondingly more power.
Fully grokking Eliezer’s post that I linked would have given you all of the above reply. The wikipedia definition is less clear than Eliezer’s post. I post here because this community is more than capable of helping/forcing me to clarify my logic and rationality.
Could someone give me a hint as to why this particular comment which was specifically in answer to a question is being downvoted? I don’t get it.
I didn’t downvote because you were right that the hypothesis I provided (there are some rational options) was not equivalent to the question (which are the rational options). This is quite a fundamental point, so extra black marks to me for being careless.
However, Einstein’s Arrogance doesn’t deal with this fundamental point, so I disagree with “would have given you all of the above reply” and still dispute its relevance to Misha’s original comment.
ETA: also you didn’t address “what is a short-term rational answer?”. Maybe these are possible reasons for downvoting?
Conditional on the weaker claim that you should join the community, option 5 is tautologically the correct one, you just happened to have phrased it in a way that sounds evil. “Do what is rational” and “do what is in your best interest” both mean “take the most effective actions to optimise your utility function”.
Semi-correct due to further information that you should infer from reality or you would have gotten explicitly with the correct answer of asking for more information. A community has certain expectations when you join it and is likely to take punitive action if you violate those expectations.
As a hint, how is 5 different from 2? Is 2 not rational or not in your best interest?
That consequence should factor into your reasoning of “what is in my best interest?”, in the same way why you don’t usually want to perform petty crimes IRL.
I read 2 as “I will perform the most rational actions among the subset of those that lead to a net positive for the community”, whereas 5 takes away everything from ‘among’ onwards.
My first question would be “How do you go about trying to fulfill the community’s metagoal?” which is very nearly the same question as “What does it mean to be a member of this community?”
But my question for you is, why do you already know what you’re eventually trying to prove when you haven’t even settled on which questions to ask yet? Data (even hypothetical data) first, conclusions after.
Your “first question” is excellent. Your question for me is even better.
What I’m trying to eventually prove is called a hypothesis. If I can disprove it, that is equally valuable to me.
Hypothesis first. Experimental design second. Then data and conclusions.
BTW, I regard “How do you go about trying to fulfill the community’s metagoal?” and “What does it mean to be a member of this community?” most simply as better phrasings of what I meant by question 5 (though the answers would probably answer 4).
I would also give 1, 2, and 3 higher priority since they are likely to be shorter answers and the answers may well permit me to join while I perform further investigation. Your questions can take a lifetime to answer and are probably answered differently by each member of the community.
This is the simplification taught in science class, that perhaps scientists even tell themselves. Really though, most of the work is forming the hypothesis, and the social process of science only manages to protect itself from the dangers of forming hypotheses without that work by having norms of collecting lots of redundant data.
The original statement said nothing about how much work each step was. In fact, the original statement was refuting a statement that was even more simplistic and strongly implied the process was limited to just data and conclusions.
I agree with your second sentence.
Strictly speaking, if the data clearly supports a conclusion, why does it matter whether you predicted the conclusion or not? Assuming your goal was to learn about the data/conclusion, not to assess your own predictive power.
Post hoc analysis is subject to all sorts of fallacies.
Huh. I just realized that there is nothing that I recognize as a clear Science/Scientific Method sequence (though there is a ton either assumed or sprinkled throughout the sequences) for me to reference.
Reread Harry Potter and the Methods of Rationality.
The piece of the sequences relevant here is probably Science as Attire.
You are not in a position to tell other people to go read the Sequences.
Here’s an even clearer phrasing that refutes that. “What is day-to-day life like for a member of this community? How does it differ from what I’m accustomed to?” Really, I’m just trying to resolve the ambiguity that Vladimir_Nesov observed. Merely knowing that a group is rational and utilitarian (or at least, that it claims to be) doesn’t narrow down what it is very much.
Also, I would find the statement of hypothesis in the original post much clearer if you said “my hypothesis is that …” What you’ve stated instead is that you’re trying to prove your hypothesis, which is, I hope, wrong—rather, you’re investigating whether your hypothesis is true, without the specific goal of proving or disproving it.
Interesting. Those were sidelights for me as well. What was definitive for me was the statement of the community’s top-most goal.
Agreed.
It’s definitive, in that it’s a definition, but it’s not very descriptive.
Too abstract, I don’t understand. Please explain the motivation and describe the question more thoroughly.
Also, upvoted because while I think this post was in error, I think it is better that buggy thinking be exposed and corrected rather than continue to be held in private. Rationality isn’t about being more right, it’s about becoming more right than you currently are, and it appears (maybe I’m wrong about this?) that mwaser has good intentions in the way of this.
Thank you. As I said below, I didn’t clearly understand the need for the explicit inclusion of motivation before. I now see that I need to massively overhaul the question and include motivation (as well as make a lot of other recommended changes).
The post has a ton of errors but I don’t understand why you think it was in error. Given that your premise about my intentions is correct, doesn’t your argument mean that posting was correct? Or, are you saying that it was in error due to the frequency of posting?
Tricky words. I meant simply that it had errors. Of course I agree that even a flawed post is useful (in that it helps to expose buggy thinking), but here it seems like you’re attempting to argue about what it means for a post to be “in error.” Taboo the word “error” and I don’t think we disagree.
Your “first question” is excellent. Your question for me is even better.
What I’m trying to eventually prove is called a hypothesis. If I can disprove it, that is equally valuable to me.
Hypothesis first. Experimental design second. Then data and conclusions.