Sunny from QAD
I really liked reading Project Lawful. Seeing a rationalist protagonist in action helped me absorb the techniques in ways that my multiple readings of the sequences never did.
Anyone have any ratfic plots they want to see written? I’m decent at writing but bad at coming up with stories.
If I brush my teeth once per day, and then immediately eat sugar afterwards every time, and I also go to the dentist once per year, is that good enough to keep my teeth clean?
I definitely agree that probabilistic rejection is a reasonable and useful strategy. I’m not saying it isn’t, I’m saying that the standard “don’t give in to threats” maxim seems to say people shouldn’t tolerate it. That puts two standard elements of our zeitgiest in conflict with each other, and I’m looking for a resolution.
If we say that rejection (even probabilistic rejection) in the Ultimatum Game is reasonable and other agents should negotiate around your tendency to do it, and we say nothing about why this doesn’t meet the definition of a threat when it prima facie seems to, then we’re saying agents should sometimes give in to threats, which we usually say they never should.
I don’t understand either of your two corrections. For #1, is it really usually a requirement that the person making you the offer be a perfect predictor? What about a human-level predictor in an iterated game? Yudkowsky writes that on dath ilan, people use The Algorithm, which is to reject unfair offers with high enough probability to make it a bad gamble for the offer-maker. Dath ilan is populated by humans, not perfect predictors, so I think it’s fair to say standard theory says that this applies to humans.
Or, well, okay, maybe I should say “the zeitgeist of Lesswrong” instead of “standard theory”. In which case, my post is challenging our zeitgeist’s understanding of these two principles, not some theorem with much stricter requirements for who it applies to.
For #2, are you sure nobody’s advocating that? I think Planecrash has a plot element that demonstrates that people do advocate this (spoilers for Planecrash, obviously):
Keltham wants to destroy the entire universe because he’d rather do that than let Hell continue to exist. He realizes that the gods may bargain him down from that by offering to just fix Hell, but ~ignores this consideration because if he fixates on that and does anything differently in pursuit of such bargaining, then he’d be making a Threat instead of just maximizing his utility function, and the gods would predictably ignore him. Carissa is around too, but he rejects several possible ways she could influence his project on the grounds that, since she doesn’t prefer to destroy the universe and only wants Hell to be fixed, then if he lets her influence him in the wrong way, the gods might consider Keltham’s universe-destruction game to ultimately be a Threat made by Carissa, and predictably ignore it.
This seems very analogous to the mugging case. If it’s different, what’s the difference? Mind you that the person making the threat is a (very smart) human, not a perfect predictor.
Alternatively, what does the zeitgiest around here say about the human-governance principle “don’t negotiate with terrorists”? That one is unambiguously applied to humans. Do we generally disagree with it for that reason?
I’m just not convinced that my post frames either of these two things incorrectly. I think people around here do generally say that human beings should both reject unfair Ultimatum Game offers and not give in to threats.
Well sure, but isn’t that still a threat?
Yes, conversion to them is what’s implied, not the ruling out of any other kind of preference. The wiki quote is basically what I had in mind.
I agree that this conversion ability is an assumption. It’s a shaky assumption to me, I haven’t thought it through to the point that I could defend it, and I’d be interested in talking about it more, especially if you disagree with it.
The reason I didn’t mention that in the top-level comment is because it’s a very standard assumption; it’s the premise that the concept of utilons (/ utility functions) are based on in general, not a feature of my specific proposal. If you think the assumption is false, then you disagree with the concept of utility functions, not with any specific proposal for what units to use when expressing them.
(Edit: btw, the specific assumption that’s standard is not just that you can convert all preferences to feelings-preferences, but actually that you can convert between any two preferences.)
I disagree-voted with your bare quote because it was a non-sequitur, and I downvoted because I found it rude.
I’m confused what you think I’m saying that rules out you having preferences that aren’t about your feelings?
Candidate definition of the utilon: 1 utilon = 1 minute of eudaimonia.
This should be your own minute of eudaimonia, unless you actually value everyone else’s at the same rate, which I think is psychologically unrealistic. Or set it to a stranger’s, and say that your own minute is worth many utilons.
I’m from dath ilan too.
I’m probably a dumber-than-average citizen of it, and I seem to have forgotten most of my training when I was dumped on Earth. But when I read Eliezer’s descriptions of what civilization was like… I recognize my home.
“But I can’t always talk like this! What about—” fine. BUT TRY.
Imagine you’re reading a fictional story, and the main character asks a kindly priest for a summary of a religion other than their own, and how their own religion differs. They might say something simple and easy to understand, like:
The Abadarans value trade above all else. They think buying and selling things, and making agreements to buy and sell things later, is the way humanity becomes wealthier.
While Irorans find trade valuable, we disagree that it is central. We feel each person’s interests are best served by that person acting alone, trading with others only when it’s necessary or obviously correct, but by no means as their default solution to all problems.
I think people should always try to talk like this.
Posts are too long. Say what you mean and fucking shut up!
I hope you find someone! Remember not to just mindlessly write “I’m lonely and don’t make connections” into your self-image. Write it if you want to write it, of course, but don’t do it mindlessly. Alternatively, write “I’m personable and I do make connections” instead, or “I’m exploring and I’m keeping my eyes peeled for connections”. And remember to be appropriately skeptical of this “knowing/assuming” business… those are really two very different verbs, and it’d be silly to write anything unhappy into your self image on the basis of a proposition that you didn’t even decide whether you know it or assume it. Good luck!
I figured! That’s what I was trying to say.
The people you personally know have a statistical skew towards being more popular compared to the background population, because more-popular people know more people.
It would be cool if I could put “Epistemic status” and “TLDR” fields at the top of a post in an actual separate UI element from the text body. They are semantically separate from the text body and I never know whether to insert a separating “—” or force myself to use a header for the section after it or what.
Log-scoring really means you’re just multiplying together the probabilities you assigned to every thing that did turn out true.
If every time you say an event has probability p, and it happens, you assign yourself log p points, you might as well just take all the p’s you’re taking logs of and multiply them together first, and then take the log after.
The logarithm is what creates addition (which is how you accumulate points in any points-based system) from multiplication (the thing probabilities want to do with each other).
I guess the reason to take the actual logarithm at the end, rather than just keeping the product of p’s, is to make the expected scores incentivize accurate predicted probabilities. But you don’t need that if all you want to do is compare who performed better at predicting a series of truths.
I don’t think being manipulatively fake-kind to someone as a ploy to enable further cruelty counts as being kind in the first place, so I don’t think it constitutes an example of “they’re choosing to be kind despite having the capacity and courage to do otherwise and have considered it”. This is someone who is able to be cruel and is in fact doing so, even though they might appear kind to the person who gets manipulated.
I want to attend a CFAR workshop someday. One note of discord in my mind is that I worry it’ll hurt to be in a room of people who are all as smart as me — probably smarter, on average — when I’m used to being the smartest one around.
I’m pretty sure there’s nothing to do or say about that except for me to get over it, but I say just for the sake of saying these things out loud.
(I don’t have the money for a workshop now anyway, so this is a problem that’s a few years out from affecting me personally.)