“A group of Singularity Institute donors has stepped forward to match all donations given through Philanthroper today”
Does this make sense? How do we know they wouldn’t have given that money anyway?
“A group of Singularity Institute donors has stepped forward to match all donations given through Philanthroper today”
Does this make sense? How do we know they wouldn’t have given that money anyway?
I’ve rethought and I was half-wrong.
It was certainly rational from SIAI’s point of view as Tiiba explained.
Also it was rational for SIAI supporters to give one of their dollars via Philanthroper as I’m pretty sure the expected extra money SIAI gets is x where $1 < x ⇐ $2.
It might not be exactly $2 because we assume the matchers have not already decided exactly how much they will give over their lifetime. Future donations from them will be somewhat based on “have I given enough already?” which would be negatively impacted by past donations.
So for people to get their full $2, matchers need to promise to match the money and not feel warm and fuzzy about it.
My understanding of the FOOM process:
An AI is developed to optimise some utility function or solve a particular problem.
It decides that the best way to go about this is to build another, better AI to solve the problem for it.
The nature of the problem is such that the best course of action for an agent of any conceivable level of intelligence is to first build a more intelligent AI.
The process continues until we reach an AI of an inconceivable level of intelligence.
This is slightly off-topic (as it doesn’t help distinguish between Yvian’s hypothesis and T&C’s) but anyway:
People who feel guilty sometimes give to charity, right?
Is the social purpose of giving (in this case) therefore to punish yourself financially rather than actually help anyone?
Realising that I was irrationally risk-averse and correcting for this in at least one major case
In terms of decision making, imagining that I had been created at this point in time, and that my past was a different person.
Utilitarian view of ethics
Actually having goals. Trying to live as if I was maximizing a utility function.
Evidence for each point:
moving from the UK to Canada to be with my girlfriend.
There is a faster bus I could have been using on my commute to work. I knew about the bus but I wasn’t taking it. Why not? I honestly don’t know. It doesn’t matter. Just take the faster bus from now on!
When I first encountered Less Wrong I tended to avoid the ethics posts, considering them lower quality. I only recently realised this was because I had been reading them as “epistemic rationality” and as a non-moral-realist they therefore didn’t seem meaningful. But as “instrumental rationality” they make a lot more sense.
This was mainly realising that “make the world a better place and look out for yourself” is somehow morally OK as a utility function. This is a very recent change for me though so no evidence yet that it’s working.
The main problem I have with mathematical notation right now is that I can’t skim it. If I am reading a document with some math notation in, I tend to just skip past it and figure out what’s going on from the surrounding text.
I can read it quickly and just see a bunch of apparently meaningless symbols. Or I can read it very, very slowly and carefully and figure out exactly what everything means and exactly what’s going on. But there’s nothing in between.
Computer code I find rather easier to skim, and natural language is much easier.
Is this a problem with the notation itself or is it just that I work with computer code on a day-to-day basis and don’t with maths, so that I’ve learnt how to skim it and spot the relevant patterns much more easily?
Good! That’s how it should be.
I was thinking about only the people who give as a response to guilt. Do you give mainly when you feel guilty, or at other times?
Also, have we addressed whether in general guilty people who self-punish feel good doing so (even as it puts them at a material/social disadvantage)?
First off, I’m using “epistemic and instrumental rationality” as defined here:
http://lesswrong.com/lw/31/what_do_we_mean_by_rationality/
If you don’t believe objective morality exists then epistemic rationality can’t be applied directly to morality. “This is the right thing to do” is not a question about the “territory” so you can’t determine its truth or falsehood.
But I choose some actions over others and describe that choice as a moral one. The place where I changed my mind is that it’s no longer enough for me to be “more moral than the average person”. I want to do the best that I can, within certain constraints.
This fits into the framework of epistemic rationality. I am essentially free to choose which outcomes I consider preferable to which. But I have to follow certain rules. Something can’t be preferable to itself. I shouldn’t switch your preference and back just by phrasing the question differently. And so on.
Another point I should elaborate on.
“Would you sacrifice yourself to save the lives of 10 others?” you ask person J. “I guess so”, J replies. “I might find it difficult bringing myself to actually do it, but I know it’s the right thing to do”.
“But you give a lot of money to charity” you tell this highly moral, saintly individual. “And you make sure to give only to charities that really work. If you stay alive, the extra money you will earn can be used to save the lives of more than 10 people. You are not just sacrificing yourself, you are sacrificing them too. Sacrificing the lives of more than 10 people to save 10? Are you so sure it’s the right thing to do?”.
“Yes”, J replies. “And I don’t accept your utilitarian model of ethics that got you to that conclusion”.
What I figured out (and I don’t know if this has been covered on LW yet) is that J’s decision can actually be rational, if:
J’s utility function is strongly weighted in favour of J’s own wellbeing, but takes everyone else’s into account too
J considers the social shame of killing 10 other people to save himself worse (according to this utility function) than his own death plus a bunch of others
The other thing I realised was that people with a utility function such as J’s should not necessarily be criticized. If that’s how we’re going to behave anyway, we may as well formalize it and that should leave everyone better off on average.
It seems plausible to me that J really, truly cares about himself significantly more than he cares about other people, certainly with P > 0.05.
The effect could be partly due to this and partly due to scope insensitivity but still… how do you distinguish one from the other?
It seems: caring about yourself → caring what society thinks of you → following society’s norms → tendency towards scope insensitivity (since several of society’s norms are scope-insensitive).
In other words: how do you tell whether J has utility function F, or a different utility function G which he is doing a poor job of optimising due to biases? I assume it would have something to do with pointing out the error and seeing how he reacts, but it can’t be that simple. Is the question even meaningful?
Re: “charities that work”, your assumption is correct.
Was the buyer sane enough to realise that it probably wasn’t a power crystal, or just sane enough to realise that if he pretended it wasn’t a power crystal he’d save $135?
Is that amount of raising-the-sanity waterline worth $135 to Tony?
I would guess it’s guilt-avoidance at work here.
(EDIT: your thanks to Tony are still valid though!)
I would expect it not to happen inside a real school because the school would be uninterested in improving their teaching via the experimental method. Of course I hope I’m wrong! (Anyone know of particularly enlightened schools?)
If I’m right it would seem a much harder problem to solve than the being-graded-fairly one.
paulfchristiano,
I’m probably one of the people you’re trying to reach. I want to help people and I am aware that the approaches favoured by society may well not work. But doing the right thing is really hard.
However many arguments you present to me it’s still really hard. For me, it’s not a lack of argument that stands in the way of doing the right thing.
What I want is a community of rationalists who are interested in helping others as much as possible. Does such a thing already exist, ready-made? Either as a subset of LW or independent of it?
I can’t help feeling that such a thing would help your cause immensely. However good your arguments are, people will want to know “what do I do next?” And they will be much happier with the answer “come meet my friends and throw some ideas around” than “give all your money to SIAI”.
Agree—interesting to see such a strong effect on people who should have been trying their hardest to ignore the anchor. (assuming some unrelated aspect of experimental design isn’t skewing the results)
Would also be interesting to have a personal quiz with many questions so that you could test your own anchoring bias. (And see if you could train yourself out of it… at least in circumstances where you know it’s an issue)
Disclaimer: didn’t take part in the experiment myself (ugh factor/fear of failing on general knowloedge quizzes)
Unrelated question: is the “if I don’t do this, downvote to −10” meme new?
Wow, anchoring! That one didn’t even occur to me!
Maybe there’s some value in creating an algorithm which accurately models most people’s moral decisions… it could be used as the basis for a “sane” utility function by subsequently working out which parts of the algorithm are “utility” and which are “biases”.
(EDIT: such a project would also help us understand human biases more clearly.)
Incidentally, I hope this “double effect” idea is based around more than just this trolley thought experiment. I could get the same result they did with the much simpler heuristic “don’t use dead bodies as tools”.
Making a better bus map of Toronto (by creating software, which will then hopefully make it reasonably straightforward to map other cities too).
Reasons:
Newly arrived in Toronto and want to contribute something to the community
Everyone I’ve told about it seems to really want it. That kind of social pressure is hard to ignore.
Biggest reason: up until now, I never seemed to be able to finish any project that I started for my own amusement. I want to get this one done just to show that I can.
I agree the authors of the paper are idiots (or seem to be—I only skimmed the paper). But the research they’re doing could still be useful, even if not for the reason they think.
I’m interested (sorry I’m a lurker so you don’t know me yet)