I took the survey.
Like several other people, I was a bit bothered by the P(God) type questions. For some of those, my belief depends on an argument for the impossibility of, say, God, rather than on any particular evidence. In that case, am I supposed to take into account my uncertainty as to the validity of my argument? Or just put 0?
It’s not so much the “genuinely omnipotent” bit that I have philosophical problems with as the idea of “ontologically basic mental entities”. I don’t think this is the place to go into it fully, but suffice it to say that nowadays I’m not sure if that even makes sense. If I don’t think a situation makes sense, how can I assign it a probability?
Of course, I could weigh that against the probability that I’m mistaken, but I’m not sure whether we’re meant to take that kind of thing into account.
Me too: if we’re in a simulation, then whoever’s running the simulation would count.
Yeah I think you’re right; I hereby retract my worries!
I think a lot of the work here is being done by the assumption that the standard deviation of the estimate error depends on (and, moreover, is linear in terms of!) the estimate itself.
Why would we assume this?
Holden suggests that this is reasonable since it keeps the probability of the “right estimate” being zero constant. This seems bizarre to me: no matter how huge my estimate is, there’s a constant chance that it’s completely off? And surely, one would expect the chance of the “correct” estimate being zero to be much higher in cases where your own estimate is close to zero.
The only other motivation I can think of is that we ought to be more suspicious of higher estimates. Hence, if we come up with a high estimate, we should be suitably suspicious and assume our estimate had a high error. But surely this “suspicion of high estimates” is precisely encoded in our prior; that is, before estimation we believe that a higher value is less likely, and our prior distribution reflects that. But then why are we adding an additional fudge factor in the form of an increased estimate error?
Even if we were convinced that we should assign higher standard deviation to the error on higher estimates, it seems far from obvious that this should be linear in our estimate!
Additionally, I’m not sure that a normal distribution is the correct distribution for healthcare charities, at least. The DCP2 data strongly suggests that the underlying distribution is log-normal. Even though DCP2 has been shown to be a lot less reliable than we’d like (here; some great work by GiveWell, if I may say so!), the sample size is large enough that we wouldn’t expect errors to change the underlying distribution.
Agreed. In particular, as a philosophy student, I would add that there is very little on LW that I would class as original philosophy. A lot of it is presented well, and in a way that is accessible to people who aren’t familiar with the language and literature of modern analytic philosophy. However, most of these issues have been discussed by analytic philosophers in some detail.
e.g. this post and the following discussion makes a pretty good effort at embedding Eliezer’s metaethical position into the usual metaethical landscape. Sure, there’s disagreement about it, but the discussion certainly can, and has been, phrased in those terms.
So I would certainly say that LW has provided me with some useful perspectives on some philosophical issues, but you can find most of the material in analytic philosophy if you look. True, there’s a whole load of tripe to wade through, but there’s something to be said for the discipline of really understanding why everyone else is wrong! And if you’re of an analytic/logical bent, you’ll probably tend to gravitate towards that part of the literature anyway.
How are you going to address the perceived and actual lack of rigor associated with [Singularity Institute]?
I upvoted this question originally, and while I appreciate your response, I don’t feel you addressed what, for me, is the crux of the matter. If the SIAI is so focussed on “solving the most important problems in mathematics, computer science, and philosophy”, then where is the progress?
The worry is that the SIAI is seen as somewhere where people pontificate endlessly about the problem, without actually doing useful work towards the solution. It is important to raise awareness of the dangers of an UFAI situation, but you’re claiming that you also want the SIAI to be more than that.
But it’s hard to take that seriously when there is so little evidence of problems actually getting solved, particularly the hard ones in mathematics and computer science. Eliezer’s TDT draft is a step in the right direction, as it’s at least evidence that some work is getting done, but it’s the sort of thing I’d like to see much, much more of. In addition, it could do with tightening up, and I think the rigour of submitting it to an actual academic journal would be extremely helpful. Even if you don’t want to do that, a public draft at least allows some kind of assessment of the work you’re doing.
As for the philosophy, I think that’s in better shape, but not an awful lot better. There’s good material in the sequences, but at the end of the day they’re a series of thoughtful blog posts, not a polished, well-structured series of arguments. The quality is better than some published philosophy, but that’s not saying much. Again, I think the discipline required to shape some of the material up to get it published would be a good thing.
As long as the SIAI continues to not publish, or otherwise make available, credible documents indicating rigorous progress it is going to be perceived as lacking in rigour. And those of us who aren’t privy to what is actually going on in there may worry that this indicates an actual lack of rigour.
So it sounds like your answer is: “Publishing research would help, and we’re working on it.”
That’s great! It’s just good that you’ve got a plan. After all, the question was “How are you going to address the perceived lack of rigour”.
Even if they were to make some actual progress, most of it would probably be regarded too dangerous to be released. Therefore I predict that you won’t see much more of it ever.
I’m not sure how true this is, but suppose it is. Then it seems to me that the SIAI has got a problem. They need people to take them seriously, in order to attract funding and researchers, but they can’t release any evidence that might make people take them seriously, as it’s regarded as “too dangerous”. Dilemma.
Secrecy and a perceived lack of rigour seem likely to go hand in hand. And for those of us outside the SIAI, who are trying to decide whether to take it seriously, said secrecy also makes it seem likely that there is an actual lack of rigour.
Perhaps this just demonstrates that any organization seriously aiming to make FAI has to be secretive, and hence have a bad public image. Which would be interesting. But in that case, the answer to the original question may just be: “We can’t really, because it would be too dangerous”, which would at least be something.
Indeed! Think about it this way, if Less Wrong would have been around for 3000 years and the field of academic philosophy would have been founded a few years ago then most of it would probably be better than Less Wrong..
And perhaps, just perhaps, LW might have something to learn from that older sibling… I appreciate the desire to declare all past philosophy diseased and start again from nothing, but I think it’s misguided. Even if you don’t like much of contemporary philosophy, modern-day philosophers are often well-trained critical thinkers, and so a bit of attention from them might help shape things up a bit.
As someone who knows a bit of advanced mathematics, I was impressed by how closely this chimed with my experience.
Okay, here are a couple of things that bother me about this whole enterprise:
Maybe this is a British thing, but it smacks a bit of Taking Oneself Too Seriously, which is a capital crime over here. For some reason the the idea of a bunch of earnest, self-ascribed “rationalists” coming up with their own rituals just makes me cringe. I’m struggling to pinpoint the source of the cringing, but it’s definitely there. It’s probably personal, but put it down as evidence that you’re going to cause adverse emotional reactions in some people.
I’m a bit wary of the idea of trying to form LW communities. Why do we want to do that? I think of LW as a great forum for discussing a bunch of intellectually interesting ideas, and I’d almost certainly be interested in meetups if there were any near me, just because discussion in person has it’s own perks. But actually trying to form meetup groups into communities? That feels a bit like mission creep to me.
Related to the above: I’m wary of setting off down a path that is likely to make LWers identify (more) strongly as being LWers. I’m with Paul Graham on this. Bundling an ideology of any kind with fun community-type stuff seems like a recipe for producing unwarranted attachment to said ideology.
I’m glad you had fun with it, but I think you could just, you know, have a party, at which you might venture to read some texts you like, or sing some songs, or not if people don’t want to. Rather than a ritual, with all the gunk associated with that.
I agree: this post felt like a self-congratulatory collection of applause lights. (I wonder if the habit of linking to previous posts every other word is an indicator of this. Most people aren’t going to follow those links, they’re just going to think, “Ah yes, a link to a Sequence post. That must be an accepted thing!”)
I also think there’s a worrying tendency towards ideology here. Luke suggests that “levelling up” in rationality led him to a bunch of beliefs, which are, coincidentally, fairly widely accepted views around here. Cue a round of back-slapping as we all congratulate ourselves on how rational we are.
But rationality doesn’t necessarily lead you anywhere: the evidence should do that. And if the evidence starts pointing somewhere else, you should move. And so I’m a bit wary of the tendency to draw too close a link between any particular beliefs and rationality. You never want to be in the situation where you’re trying to persuade someone of your views and you find yourself saying “But it’s the rational thing to believe!” instead of presenting the evidence.
Also: hints of the No True Scotsman fallacy.
When genuine curiosity tore down those walls, it didn’t take long for the implications of my atheism to propagate.
Woe betide you who don’t come to the same conclusions as Luke: your curiousity clearly isn’t genuine!
Now this may all sound a bit harsh, but frankly I really wish Luke would stop writing posts like this and start doing some hard-headed thinking about some actual problems.
This seems to be true only if your mental states of knowledge can have no effect on things that you care about… that is to say, it’s pretty clearly false!
Sometimes humans care about their own mental states. It might count positively in my utility function that I not be psychologically distressed. Plausibly, learning certain bits of knowledge that I can’t use practically (“The Great Old Ones are about to awake and devour the earth!”) might have this effect with no compensating benefits, and so learning them would have negative utility.
Alternatively, my state of knowledge may affect something in the world. If I’m very bad at concealing my knowledge, then knowing about the secret conspiracy to take over the world may just get me killed: that’s pretty clearly negative utility. Or, say, on being told that the fate of the world rests upon my performance in some task, I’m likely to do worse than if I didn’t know (due to nerves).
In short: if you’re a disembodied observer who merely picks effects to occur in the world, and whose utility function ranges strictly over things in the world other than yourself, then this might be true. Otherwise, not so much.
You constantly elide between the property of being green and the experience of something green. Which leads to the ancient mistake of saying that whatever constitutes your experience of something green must itself be green. Admittedly you put this enormous red herring in the mouth of your opponent, but it’s totally unwarranted nonetheless.
e.g.
Such a theory will necessarily present a candidate, however vague, for the physical correlate of an experience of color. One can then say that color exists without having to add anything to physics, because the color just is the proposed physical correlate.
The happy materialist might say: but those aren’t the things which are truly green in the sense you care about; the things which are green are parts of experiences, not the external objects.
You also then essentially just say “But qualia! Intentionality! They’re so real! There must be something more!”, i.e. the same argument dualists have been making since the dawn of time, and that any attempts to dissolve the question have failed, since
all you have to do is attend for a moment to experience itself, and then to compare that to the picture of billions of colorless atoms in intricate motion through space, to realize that this is still dualism.
Furthermore, all the arguements you use are pretty much applicable across the board, and don’t particularly relate to functionalism, so I think it’s disingenuous of you to say that you’re arguing for “functionalism implies dualism” rather than simply “dualism is true”.
Downvoted.
it’s a bias if it keeps you from achieving your other values.
This is nonsense. I have a desire for ice cream, but also a desire to stick to my diet and lose weight. Oh no, my desire to stick to my diet is preventing my from achieving my desire for ice cream, it must be a bias!
This may be true, and so we might expect someone who was very wealthy to lose their risk-aversion for deicisions where they were sure there was no risk of losses cutting into what they need for personal use. Sounds pretty reasonable for a risk-averse agent to me.
This! If you’re risk averse, then you want to avoid risk, and so in the real utility calculation upon which you base your decisions the risk-averse option gets a little extra positive term for being, well, risk-averse. And then the two options no longer have the same expected utility.
That’s really not how I would understand a bias. I would think of a bias as a feature of your psychology that distorts your decision-making process away from the rational; that is, optimal pursuit of your goals. The planning fallacy is a bias, having conflicting goals is just a feature of my utility function.
I don’t think that follows. The risk-aversion utility attaches to the choice, not the outcome: I get extra utility for having made a choice with lower expected variance, not for one of the outcomes. If you then offer me a choice between choices, then sure, there will be more risk aversion, but I don’t think it’s viciously recursive.
I think this is relevant to some of the discussion here: David Chalmers, The Matrix as Metaphysics.
Chalmers argues that discovering that the simulation hypothesis was true would not show our beliefs about the existence of physical objects to be false, but merely indicate a surprising truth about the nature of reality. So he would say that yes, the rock does exist even if the simulation hypothesis is true.