Could it be that the average customer hasn’t thought it through enough to realize they are incinerating $1.67 of time-value, and would thus prefer to pay $15 plus *mumble* time as opposed to $15.25 plus zero time?
Ninety-Three
Karma: 434
I got all of the octopus questions right (six recalled facts, #6 intuitively plausible, #9 seems rare enough that it should be unlikely for humans to observe, and #2 was uncertain until I completed the others then metagamed that a 7⁄2 split would be “too unbalanced” for a handcrafted test) so the only surprising fact I have to update on is that the recognition thing is surprising to others. My model was that many wild animals are capable of recognizing humans, and octopuses are particularly smart as animals go, no other factors weigh heavily. That octopuses evolved totally separated from humans didn’t seem significant because although most wild animals were exposed to humans I see no obvious incentive for most of them to recognize individual humans, so the cases should be comparable on that axis. I also put little weight on octopuses not being social creatures because while there may be social recognition modules, A: animals are able to recognize humans and all of them generalizing their social modules to our species seems intuitively unlikely and B: At some level of intelligence it must be possible to distinguish individuals based on sheer general pattern-recognition, for ten humans an octopus would only need four or five bits of information and animal intelligence in general seems good at distinguishing between a few totally arbitrary bits.
The evolutionary theory of aging is interesting and seems to predict that an animal’s maximum age will be proportionate to its time -to-accidental-death. Just thinking of animals and their ages at random this seems plausible but I’m hardly being rigorous, have there been proper analyses done of that?
“it is impossible for there to be a language in which most sentences were lies”
Is it? If 40% of the time people truthfully described what colour a rock was, and 60% of the time they picked a random colour to falsely describe it as (perhaps some speakers benefit from obscuring the rock’s true colour but derive no benefit from false belief in any particular colour), we would have a case where most sentences describing the rock were lies and yet listening to someone describing an unknown rock still allowed you to usefully update your priors. That ability to benefit from communication seems like all that should be necessary for a language to survive.
I am opposed to the implementation of this exercise, I believe its basic concept seriously undercuts the moral lesson we should take from Petrov Day.
The best way to not blow ourselves up is to not make nuclear weapons. On a day dedicated to not blowing ourselves up, LW has decided to manufacture a bunch of completely unneeded nuclear weapons, hand them out to many people, and then hope really hard that no one uses them. This is like a recovering addict carrying drugs on his person in order to make a point about resisting temptation: he is at best bragging and at worst courting disaster so boldly that one should wonder if he really wants to avoid self-destruction. This makes a good allegory for the senseless near-apocalypse of the Cold War, but deliberately creating a senseless risk does not seem like an appropriate way of celebrating the time we narrowly avoided triggering a senseless risk.
“I don’t care what future people think of my morality, I just care what’s moral by the arbitrary standards of the time I live in.”
As a moral super-anti-realist (“Morality is a product of evolutionary game theory shaping our brains plus arbitrary social input”) this doesn’t represent my view.
I care about morality the same way I care about aesthetics: “I guess brutalism, rock music and prosocial behaviour are just what my brain happens to be fond of, I should go experience those if I want to be happy.” I think this is heavily influenced by the standards of the time, but not exactly equal to those standards, probably because brains are noisy machines that don’t learn standards perfectly. For instance, I happen to think jaywalking is not immoral so I do it without regard for how local standards view jaywalking.
Concisely, I’d phrase it as “I don’t care what future people think of my morality, I just care what’s moral by the arbitrary standards of my own brain.”
As an exercise in describing hard takeoff using only known effects, this story handwaves the part I always had the greatest objection to: What does Clippy do after pwning the entire internet? At the current tech level, most of our ability to manufacture novel new goods is gated behind the physical labour requirements of building factories: even supposing you could invent grey goo from first principles plus publicly available research, how are you going to build it?
A quiet takeover could plausibly use crypto wealth to commission a bunch of specialized equipment to get a foothold in the real world a month later when it’s all assembled, but going loud as Clippy did seems like it’s risking a substantial chance that the humans successfully panic and Shut. Down. Everything.
As a lurker, I failed to understand this system in a way that led to me completely ignoring it (I probably would have engaged more with LW this week had I understood, having noticed now it feels too late to bother), so I feel like I should document what went wrong for me.
I read several front-page posts about the system but did not see this one until today. The posts I read were having fun with it rather than focusing on communication, plus the whole thing was obviously an extended April Fool’s joke so I managed to come away with a host of misconceptions including total ignorance of the core “no really, karma equals actual money for you” feature. I assumed that if it was serious people would be trying a lot harder to communicate the incentives to people (compare announcements of LW bounties, which I routinely manage to hear about even in periods where I’ve fallen out of the habit of checking this website).
On top of “karma equals money” being fundamentally implausible, an April 1st joke named Good Heart Tokens feels like it was designed to not be taken seriously. If the system was meant to incentivize posts from lurkers, more effort could have been put into making the incentives clear.
Edit: Making this comment I double-checked some things and thought I came to a fully correct understanding of the system but upon hitting submit I became confused again. This post says that self-votes don’t count but my fresh comment displays as having 1 token. There is a token counter on the user profile page, but as far as I can tell from looking at the pages of a few random users, that counter is tracking neither karma nor any calculation I can imagine representing token count, I have no idea what it’s doing.
The numbers attached to posts and comments seem to be a straightforward reskin of karma: includes self-votes, does not include the 3x multiplier for posts. The token counter in user profiles seems to update instantly (I tried voting up and down on posts and comments then refreshing the page of the user to test) but undercounts in ways I don’t understand. For instance, this random user currently displays as 207
karma(edit: tokens, not karma, doh) based on a post with 68 karma and about a dozen comments with net karma-minus-selfvotes of ~15. I can tell it’s up to date because it went up by 3 when I upvoted his post, but it seems like it ought to be ~216 (post karma times three plus comment karma) and I can’t explain the missing tokens, and several of the random users I looked at displayed this sort of obvious undercounting.
Looking at more random users, I think tokens earned via posting are being undercounted somehow. Users with only comments display as having exactly the amount of tokens I would expect from total karma on eligible posts minus self-votes, but looking at users with posts made after April 2 (to avoid complications from a changing post-value formula) consistently have less than the “comment votes plus 3x post votes, not counting self votes” formula would predict. For instance, Zvi has two posts (currently 52 and 68 karma) and zero comments in the last week. With strength 2 selfvotes, (52-2+68-2)*3=348 expected tokens which is a significant mismatch from his displayed 302. It doesn’t seem to be out of date since his displayed tokens change instantly in response to me voting on the posts, is something going wrong or is there some weird special case way of voting on posts that doesn’t get immediately reflected in the user page?
Doh. I forget how much faster strong upvotes scaled with user karma, that resolves all my confusion.
At least, it defects if that’s all there is to the world. It’s technically possible for an LDT agent to think that the real world is made 10% of cooperate-rocks and 90% opponents who cooperate in a one-shot PD iff their opponent cooperates with them and would cooperate with cooperate-rock, in which case LDT agents cooperate against cooperate-rock.
If we’re getting this technical, doesn’t the LDT agent only cooperate with cooperate-rocks if all of the above and if “would cooperate with cooperate-rock” is a quality opponents can learn about? The default PD does not give players access to their opponent’s decision theory.
The positive votes are described as “Good”, “Quite important” and “Extremely important” while the negative votes are described as “Misleading, harmful or unimportant”, “Very Misleading, harmful or unimportant” and “Highly Misleading, harmful or unimportant”. It may be unwise to change the labels halfway through voting, but I am noting for the future that the positive vs negative labels are inconsistent and this seems suboptimal. If +9 is extremely important, surely −9 should be extremely unimportant?
> Aim for convergence on truth, and behave as if your interlocutors are also aiming for convergence on truth.
It’s not clear to me what the word “convergence” is doing here. I assume the word means something, because it would be weird if you had used extra words only to produce advice identical to “Aim for truth, and behave as if your interlocutors are also aiming for truth”. The post talks about how truthseeking leads to convergence among truthseekers, but if that were all there was to it then one could simply seek truth and get convergence for free. Apparently we ought to seek specifically convergence on truth, but what does seeking convergence look like?
I’ve spent a while thinking on it and I can’t come up with any behaviours that would constitute aiming for truth but not aiming for convergence on truth, could you give an example?
Everyone sometimes issues replies that are not rebuttals, but there is an expectation that replies will meet some threshold of relevance. Injecting “your comment reminds me of the medieval poet Dante Alighieri” into a random conversation would generally be considered off-topic, even if the speaker genuinely was reminded of him. Other participants in the conversation might suspect this speaker of being obsessed with Alighieri, and they might worry that he was trying to subvert the conversation by changing it to a topic no one but him was interested in. They might think-but-be-too-polite-to-say “Dude, no one cares, stop distracting from the topic at hand”.
The behaviour Raemon was trying to highlight is that you soapbox. If it is line with your values to do so, it still seems like choosing to defect rather than cooperate in the game of conversation.
I agree that people hearing Zack say “I think this is insane” will believe he has a lower P(this is insane) than people hearing him say “This is insane”, but I’m not sure that establishes the words mean that.
If Alice goes around saying “I’m kinda conservative” it would be wise to infer that she is probably conservative. If Bob goes around saying “That’s based” in the modern internet sense of the term, it would also be wise to infer that he is probably a conservative. But based doesn’t mean Bob is conservative, semantically it just means something like “cool”, and then it happens to be the case that this particular synonym for cool is used more often by conservatives than liberals.
If it turned out that Alice voted party line Democrat and loved Bernie Sanders, one would have a reasonable case that she had used words wrong when she said she was kinda conservative, those words mean basically the opposite of her circumstances. If it turned out that Bob voted party line Democrat and loved Bernie Sanders, then one might advise him “your word choice is causing people to form a false impression, you should maybe stop saying based”, but it would be weird to suggest this was about what based means. There’s just an observable regularity of our society that people who say based tend to be conservative, like how people who say “edema” tend to be doctors.
If Zack is interested in accurately conveying his level of confidence, he would do well to reserve “That’s insane” for cases where he is very confident and say “That seems insane” when he is less confident. If he instead decided to use “That’s insane” in all cases, that would be misleading. But I think it is significant that this would be a different kind of misleading than if he were to use the words “I am very confident that is insane”, even if the statements cause observers to make the exact same updates.
Is it wrong for Bob the Democrat to say “based” because it might lead people to incorrectly infer he is a conservative? Is it wrong for Bob the plumber to say “edema” because it might lead people to incorrectly infer he is a a doctor? If I told Bob to start saying “swelling” instead of “edema” then I feel like he would have some right to defend his word use: no one thinks edema literally means “swelling, and also I am a doctor” even if they update in a way that kind of looks like it does.
I don’t think we have a significant disagreement here, I was merely trying to highlight a distinction your comment didn’t dwell on, about different ways statements can be perceived differently. “There is swelling” vs “There is swelling and also I am a doctor” literally means something different while “There is swelling” vs “There is edema” merely implies something different to people familiar with who tends to use which words.
I understood “based” to be a 4chan-ism but I didn’t think very hard about the example, it is possible I chose a word that does not actually work in the way I had meant to illustrate. Hopefully the intended meaning was still clear.
Here is a possible defense of pre-truth. I’m not sure if I believe it, but it seems like one of several theories that fit the available evidence.
Willingness to lie is a generally useful business skill. Businesses that lie to regulators will spend less time on regulatory compliance, businesses that lie to customers will get more sales, etc. The optimal amount of lying is not zero.The purpose of the pre-truth game is to allow investors to assess the founder’s skill at lying, because you wouldn’t want to fund some chump who can’t or won’t lie to regulators. Think of it as an initiation ritual: if you run a criminal gang it might be useful to make sure all your new members are able to kill a man, and if you run a venture capital firm it might be useful to make sure all the businessmen you invest in are skilled liars. The process generates value in the same way as any other skill-assessing job interview. There’s a conflict which features lying, but it’s a coalition of founders and investors against regulators and customers.
So why keep the game secret? Well it would probably be bad for the startup scene if it became widely known that everyone’s hoping startups will lie to regulators and customers. Also, by keeping the game secret you make “figure out what game we’re playing” a part of the interview process, and you’d probably prefer to invest in people savvy enough to figure that out on their own.
Depending on exactly where the boundaries of the pre-truth game are, I think I could argue no one is being deceived (I mean realistically there will be at least a couple naive investors who think founders are speaking literal truth, but there could be few enough that hoodwinking them isn’t the point).
When founders present a slide deck full of pre-truths about how great their product is, that slide deck is aimed solely at investors. The founder usually doesn’t publish the slide deck, and if they did they wouldn’t expect Joe Average to care much. The purpose of the pre-truths isn’t to make anyone believe that their product is great (because all the investors know that this is an audition for lying, so none of them are going to take the claims literally), rather it is to demonstrate to investors that the founder is good at exaggerating the greatness of their product. This establishes that a few years later when they go to market, they will be good at telling different lies to regulators, customers, etc.
The pre-truth game could be a trial run for deceiving people, rather than itself being deceptive.
If you’re not saying to go into AI safety research, what non-business-as-usual course of action are you expecting? Is your premise that everyone taking this seriously should figure out their comparative advantage within an AI risk organization because they contain many non-researcher roles, or are you imagining some potential course of action outside of “Give your time/money to MIRI/HCAI/etc”?