I interpreted the name as meaning “performed free association until the faculty of free association was exhausted”. It is, of course, very important that exhausting the faculty does not guarantee that you have exhausted the possibility space.
Dweomite
Karma: 1,445
Alas, unlike in cryptography, it’s rarely possible to come up with “clean attacks” that clearly show that a philosophical idea is wrong or broken.
I think the state of philosophy is much worse than that. On my model, most philosophers don’t even know what “clean attacks” are, and will not be impressed if you show them one.
Example: Once in a philosophy class I took in college, we learned about a philosophical argument that there are no abstract ideas. We read an essay where it was claimed that if you try to imagine an abstract idea (say, the concept of a dog), and then pay close attention to what you are imagining, you will find you are actually imagining some particular example of a dog, not an abstraction. The essay went on to say that people can have “general” ideas where that example stands for a group of related objects rather than just for a single dog that exactly matches it, but that true “abstract” ideas don’t exist.[1]
After we learned about this, I approached the professor and said: This doesn’t work for the idea of abstract ideas. If you apply the same explanation, it would say: “Aha, you think you’re thinking of abstract ideas in the abstract, but you’re not! You’re actually thinking of some particular example of an abstract idea!” But if I’m thinking of a particular example, then there must be at least one example to think of, right? So that would prove there is at least one member of the class of abstract ideas (whatever “abstract ideas” means to me, inside my own head). Conversely, if I’m not thinking of an example, then the paper’s proposed explanation is wrong for the idea of abstract ideas itself. So either way, there must be at least one idea that isn’t correctly explained by the paper.
The professor did not care about this argument. He shrugged and brushed it off. He did not express agreement, he did not express a reason for disagreement, he was not interested in discussing it, and he did not encourage me to continue thinking about the class material.
On my model, the STEM fields usually have faith in their own ideas, in a way where they actually believe those ideas are entangled with the Great Web. They expect ideas to have logical implications, and expect the implications of true ideas to be true. They expect to be able to build machines in real life and have those machines actually work. It’s something like taking ideas seriously, and something like taking logic seriously, and taking the concept of truth seriously, and seriously believing that we can learn truth if we work hard. I’m not sure if I’ve named it correctly, but I do think there’s a certain mental motion of genuine truth-seeking that is critical to the health of these fields and that is much less common in many other fields.
Also on my model, the field of philosophy has even less of this kind of faith than most fields. Many philosophers think they have it, but actually they mostly have the kind of faith where your subconscious mind chooses to make your conscious mind believe a thing for non-epistemic reasons (like it being high-status, or convenient for you). And thus, much of philosophy (though not quite all of it) is more like culture war than truth-seeking (both among amateurs and among academics).
I think if I had made an analogous argument in any of my STEM classes, the professor would have at least taken it seriously. If they didn’t believe the conclusion but also couldn’t point out a specific invalid step, that would have bothered them.
I suspect my philosophy professor tagged my argument as being from the genre of math, rather than the genre of philosophy, then concluded he would not lose status for ignoring it.
- ^
I think this paper was clumsily pointing to a true and useful insight about how human minds naturally tend to use categories, which is that those categories are, by default, more like fuzzy bubbles around central examples than they are like formal definitions. I suspect the author then over-focused on visual imagination, checked a couple of examples, and extrapolated irresponsibly to arrive at a conclusion that I hope is obviously-false to most people with STEM backgrounds.
- ^
An awful lot of people, probably a majority of the population, sure do feel deep yearning to either inflict or receive pain, to take total control over another or give total control to another, to take or be taken by force, to abandon propriety and just be a total slut, to give or receive humiliation, etc.
This is rather tangential to the main thrust of the post, but a couple of people used a react to request a citation for this claim.
One noteworthy source is Aella’s surveys on fetish popularity and tabooness. Here is an older one that gives the % of people reporting interest, and here is a newer one showing the average amount of reported interest on a scale from 0 (none) to 5 (extreme), both with tens of thousands of respondents.
Very approximate numbers that I’m informally reading off the graphs:
Giving pain: 30% of people interested (first graph), 2⁄5 average interest (second graph)
Receiving pain: 35% and 2⁄5
Being dominant: 30% and 3⁄5
Being submissive: 40% and 3⁄5
Rapeplay: >10% giving, 20% receiving, the second graph combines these at 2⁄5
Slut Humiliation (first graph): 25%
Humiliation (second graph): 2⁄5
Note that a 3⁄5 average interest could mean either that 60% of people are extremely into it or that nearly everyone is moderately into it (or anything in between). Which seems to imply the survey used in the more recent graph has significantly kinkier answers overall, unless I’m misunderstanding something. (I’m fairly certain that people with zero interest ARE being included in the average, because several other fetishes have average interest below 1, which should be impossible if not.)
If we believe this data, it seems pretty safe to guess that a majority of people are into at least one of these things (unless there is near-total overlap between them). The claim that a majority “feel a deep yearning” is not strongly supported but seems plausible.
(I was previously aware that BDSM interest was pretty common for an extremely silly reason: I saw some people arguing about whether or not Eliezer Yudkowsky was secretly the author of The Erogamer, one of them cited the presence of BDSM in the story as evidence in favor, and I wanted to know the base rate to determine how to weigh that evidence.
I made an off-the-cuff guess of “between 1% and 10%” and then did a Google search with only mild hope that this statistic would be available. I wasn’t able today to re-find the pages I found then, but according to my recollection, my first search result was a page describing a survey of ~1k people claiming a ~75% rate of interest in BDSM, and my second search result was a page describing a survey of ~10k people claiming ~40% had participated in some form of BDSM and an additional ~40% were interested in trying it. I was also surprised to read (on the second page) that submission was more popular than dominance, masochism was more popular than sadism, and masochism remained more popular than sadism even if you only looked at males. Also, bisexuality was reportedly something like 5x higher within the BDSM-interested group than outside of it.)
If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.
I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/mental state”, then why would the things humans actually value necessarily be the same as Goodness?
What would you want such a disclaimer or hint to look like?
(I am concerned that if a post says something like “this post is aimed at low-level people who don’t yet have a coherent foundational understanding of goodness and values” then the set of people who actually continue reading will not be very well correlated with the set of people we’d like to have continue reading.)
A smart human-like mind looking at all these pictures would (I claim) assemble them all into one big map of the world, like the original, either physically or mentally.
On my model, humans are pretty inconsistent about doing this.
I think humans tend to build up many separate domains of knowledge and then rarely compare them, and even believe opposite heuristics by selectively remembering whichever one agrees with their current conclusion.
For example, I once had a conversation about a video game where someone said you should build X “as soon as possible”, and then later in the conversation they posted their full build priority order and X was nearly at the bottom.
In another game, I once noticed that I had a presumption that +X food and +X industry are probably roughly equally good, and also a presumption that +Y% food and +Y% industry are probably roughly equally good, but that these presumptions were contradictory at typical food and industry levels (because +10% industry might end up being about 5 industry, but +10% food might end up being more like 0.5 food). I played for dozens of hours before realizing this.
I don’t think Eliezer’s actual real-life predictions are narrow in anything like the way Klurl’s coincidentally-correct examples were narrow.
Also, Klurl acknowledges several times that Trapaucius’ arguments do have non-zero weight, just nothing close to the weight they’d need to overcome the baseline improbability of such a narrow target.
Thank you for being more explicit.
If you write a story where a person prays and then wins the lottery as part of a demonstration of the efficacy of prayer, that is fictional evidence even though prayer and winning lotteries are both real things.
In your example, it seems to me that the cheat is specifically that the story presents an outcome that would (legitimately!) be evidence of its intended conclusion IF that outcome were representative of reality, but in fact most real-life outcomes would have supported the conclusion much less than that. (i.e. there are many more people who pray and then fail to win the lottery, than there are people who pray and then do win.)
If you read a story where someone tried and failed to build a wooden table, then attended a woodworking class, then tried again to build a table and succeeded, I think you would probably consider that a fair story. Real life includes some people who attend woodworking classes and then still can’t build a table when they’re done, but the story’s outcome is reasonably representative, and therefore it’s fair.
Notice that, in judging one of these fair and the other unfair, I am relying on a world-model that says that one (class of) outcome is common in reality and the other is rare in reality. Hypothetically, someone could disagree about the fairness of these stories based only on having a different world-model, while using the same rules about what sorts of stories are fair. (Maybe they think most woodworking classes are crap and hardly anyone gains useful skills from them.)
But I do not think a rare outcome is automatically unfair. If a story wants to demonstrate that wishing on a star doesn’t work by showing someone who needs a royal flush, wishes on a star, then draws a full house (thereby losing), the full house is an unlikely outcome, but since it’s unlikely in a way that doesn’t support the story’s aesop, it’s not being used as a cheat. (In fact, notice that every exact set of 5 cards they might have drawn was unlikely.)
If your concern is that Klurl and Trapaucius encountered a planet that was especially bad for them in a way that makes their situation seem far more dangerous than was statistically justified based on the setup, then I think Eliezer probably disagrees with you about the probability distribution that was statistically justified based on the setup.
If, instead, your concern is that the correspondence between Klurl’s hypothetical examples and what they found when reaching the planet was improbably high, then I agree that is very coincidental, but I do not think that coincidence is being used as support for the story’s intended lessons. The story is not trying to convince you that Klurl can narrowly predict exactly what they’ll find, and in fact Klurl denies this several times.
The coincidence could perhaps cause some readers to conclude a high degree of predictability anyway, despite lack of intent. I’d consider that a bad outcome, and my model of Eliezer also considers that a bad outcome. I’m not sure there was a good way to mitigate that risk without some downside of equal or greater severity, though. I think there’s pedagogical value in pointing out a counter-example that is familiar to the reader at the time the argument is being made, and I don’t think any simple change to the story would allow this to happen without it being an unlikely coincidence.
I notice I am confused about nearly everything you just said, so I imagine we must be talking past each other.
On the contrary: This is perhaps the only way the story could avoid generalizing from fictional evidence. Your complaint about Klurl’s examples are that they are “coincidentally” drawn from the special class of examples that we already know are actually real, which makes them not fictional. Any examples that weren’t special in this way would be fictional evidence, and readers could object that we’re not sure if those examples are actually possible.
If you think that the way the story played out was misleading, that seems like a disagreement about reality, not a disagreement about how stories should be used. Any given story must play out in one particular way, and whether that one way is representative or unrepresentative is a question of how it relates to reality, not a question of narrative conventions. If Trapaucius had arrived at the planet to find Star Trek technology and been immediately beamed into a holding cell, would that somehow have been less of a cheat, because it wasn’t real?
I would agree that, while reality-in-general has a surprising amount of detail, some systems still have substantially more detail than others, and this model applies more strongly to systems with more detail. I think of computer-based systems as being in a relatively-high-detail class.
I also think there are things you can choose to do when building a system to make it more durable, and so another way that systems vary is in how much up-front cost the creator paid to insulate the system against entropy. I think furniture has traditionally fallen into a high-durability category, as an item that consumers expect to be very long-lived...although I think modernity has eroded this tradition somewhat.
I have a tentative model for this category of phenomenon that goes something like:
Reality has a surprising amount of detail. Everyday things that you use all the time and appear simple to you are actually composed of many sub-parts and sub-sub-parts all working together.
The default state of any sub-sub-part is to not be in alignment with your purpose. There are many more ways for a part to be badly-aligned than for it to be well-aligned, so in order for it to be aligned, there has to be (at some point) some powerful process that selectively makes it be aligned.
Even if a part was aligned, the general nature of entropy means there are many petty, trivial reasons that it could stop being aligned with little fanfare. (Though the mean-time-to-misalignment can vary dramatically depending on which part we’re talking about.)
So, it shouldn’t be surprising when find that a complex system is broken in seven different ways for trivial and banal reasons. That’s the default outcome if you just put a system in a box and leave it there for a while.
OK, but if that’s the default state, then how do I explain the systems that aren’t like that?
Suppose we have a system that is initially working perfectly until, one day, one tiny thing goes wrong with the system.
If people use the system frequently and care about the results, then someone will promptly notice that there is one tiny thing wrong.
If the person who discovers this expects to continue using the system in the future, they have an incentive to fix the problem.
If there is only one problem, and it is tiny, then the cost to diagnose the fix the problem is probably small.
So, very often, the person will just go ahead and fix it, immediately and at their own expense, just to make the problem go away.
No one keeps careful track of this—not even the person performing the fix. So this low-level ongoing maintenance fades into the background and gets forgotten, creating the illusion of a system that just continues working on its own.
This is especially true for multi-user systems where no individual user does a large percentage of the maintenance
I don’t think this invisible-maintenance situation describes the majority of systems, but I think it does describe the majority of user-system interactions, because the systems that get this sort of maintenance tend to be the ones that are heavily used. This creates the illusion that this is normal.
Some of the ways this can fail include:
Users cannot tell that the system has developed a small problem
Maybe the system’s performance is too inconsistent for a small problem to be apparent
Maybe the operator is not qualified to judge the quality of the output
Maybe the system is used so infrequently that there’s time for several problems develop between uses
For an individual user, the cost (to that user) of fixing a problem is higher than the selfish benefits to that particular user of the problem being fixed
Maybe no single user expects to use the system very many times in the future
Maybe users lack the expertise or the authority to perform the fix (and there is no standard channel for maintenance requests that is sufficiently cheap and reliable)
Maybe the system is just inherently expensive to repair or to debug (relative to the value the system provides to a single user)
On my reading, most of Klurl’s arguments are just saying that Trapaucius is overconfident. Klurl gives many specific examples of ways things could be different than Trapaucius expects, but Klurl is not predicting that those particular examples actually will be true, just that Trapaucius shouldn’t be ruling them out.
“I don’t recall you setting an exact prediction for fleshling achievements before our arrival,” retorted Trapaucius.
“So I did not,” said Klurl, “but I argued for the possibility not being ruled out, and you ruled it out. It is sometimes possible to do better merely by saying ‘I don’t know’”
Eliezer chooses to use many specific examples that do happen to be actually true, which makes Klurl’s guesses extremely coincidental within the story. This is bad for verisimilitude, but reduces the difficulty to the reader in understanding the examples, and makes a clearer and more water-tight case that Trapaucius’ arguments are logically unsound.
Are not speculative arguments about reality normally shelved as nonfiction?
First to superintelligence wins
This phrasing seems ambiguous between the claims “the first agent to BE superintelligent wins” and “the first agent to CREATE something superintelligent wins”.
This distinction might be pretty important to your strategy.
Thoughts that occurred...
There is a cognitive cost associated with tracking echoes, which increases the more you track
Expectations about how many echoes you track are at least partly a negotiation over how labor should be distributed; e.g. am I responsible for mitigating the emotional damage that I take from your opinions, or are you?
Skills (and related issues) make this cost higher for some people and lower for others
People may have misunderstandings about what costs others are actually paying, or being expected to pay
The ability to predict echoes can be used in a friendly way (e.g. to make the conversation more comfortable for the other person) but can also be used in an unfriendly way (e.g. to manipulate them, such as in the example of asking parents if a friend can stay, in front of that friend)
This reminds me of Social status part 1/2: negotiations over object-level preferences, particularly because of your comment that Japan might develop a standard of greater subtlety because they can predict each other better.
Among other points in the essay, they have a model of “pushiness” where people can be more direct/forceful in a negotiation (e.g. discussing where to eat) to try to take more control over the outcome, or more subtle/indirect to take less control.
They suggest that if two people are both trying to get more control they can end up escalating until they’re shouting at each other, but that it’s actually more common for two people to both be trying to get less control, because the reputational penalty for being too domineering is often bigger than whatever’s at stake in the current negotiation, and so people try to be a little more accommodating than necessary, to be “on the safe side”, and this results in people spiraling into indirection until they can no longer understand each other.
They suggested that more homogenized cultures can spiral farther into indirection because people understand each other better, while more diverse cultures are forced to stop sooner because they have more misunderstandings, and so e.g. the melting-pot USA ends up being more blunt than Japan.
They also suggested that “ask culture” and “guess culture” can be thought of as different expectations about what point on the blunt/subtle scale is “normal”. The same words, spoken in ask culture, could be a bid for a small amount of control, but when spoken in guess culture, could be a bid for a large amount of control.
I’m quite glad to be reminded of that essay in this context, since it provides a competing explanation of how ask/guess culture can be thought of as different amounts of a single thing, rather than two fundamentally different things. I’ll have to do some thinking about how these two models might complement or clash with each other, and how much I ought to believe each of them where they differ.
The paper actually includes a second experiment where they had observers watch a video recording of a conversation and say whether they thought the person on the video was flirting. Results in table 4, page 15; copied below, but there doesn’t seem to be a way to format them as a table in a LessWrong comment:
Observer | Target | Flirting conditions | Accuracy (n)
Female Female Flirting 51% (187)
Female Female Non-flirting 67% (368)
Female Male Flirting 22% (170)
Female Male Non-flirting 64% (385)
Male Female Flirting 43% (76)
Male Female Non-flirting 68% (149)
Male Male Flirting 33% (64)
Male Male Non-flirting 62% (158)Among third-party observers, females observing females had the highest accuracy, though their perception of flirting is still only 18 percentage points higher when flirting occurs than when it doesn’t.
Third-party observers in all categories had a larger bias towards perceiving flirting than the people who were actually in the conversation. Though this experimental setup also had a larger percentage of people actually flirting, so this bias was actually reasonably accurate to the data they were shown.
Though, again, this study looks shoddy and should be taken with a lot of salt.
I’m confused by the study you cited. It seems to say that 14 females self-reported as flirting and that “18% (n = 2)” of their partners correctly believed they were flirting, but 2⁄14 = 14% and 3⁄14 = 21%. To get 18% of 14 would mean about 2.5 were right. Maybe someone said “I don’t know” and that counted that as half-correct? If so, that wasn’t mentioned in the procedure section.
It also says that 11 males self-reported as flirting, and lists accuracy as “36% (n = 5)”, but 5⁄11 would be 45%; an accuracy of 36% corresponds to 4⁄11.
I don’t think I trust this paper’s numbers.
If we were to take the numbers at face value, though, the paper is effectively saying that female flirting is invisible. 18% correctly believed the girls were flirting when they were, but 17% believed they were flirting even when they weren’t, and with only 14 girls flirting, 1% is a rounding error. So this is saying that actual female flirting has zero effect on whether her partner perceives her as flirting.
“Possible” is a subtle word that means different things in different contexts. For example, if I say “it is possible that Angelica attended the concert last Saturday,” that (probably) means possible relative to my own knowledge, and is not intended to be a claim about whether or not you possess knowledge that would rule it out.
If someone says “I can(not) imagine it, therefore it’s (not) possible”, I think that is valid IF they mean “possible relative to my understanding”, i.e. “I can(not) think of an obstacle that I don’t see any way to overcome”.
(Note that “I cannot think of a way of doing it that I believe would work” is a weaker claim, and should not be regarded as proof that the thing is impossible even just relative to your own knowledge.)
If that is what they mean, then I think the way to move forward is for the person who imagines it impossible to point out an obstacle that seems insurmountable to them, and then the person who imagines it possible to explain how they imagine solving it, and repeat.
If someone is trying to claim that their (in)ability to imagine something means that the laws of the universe (dis)allow it, then I think the person imagining it is impossible had better be able to point out a specific conflict between the proposal and known law, and the person imagining it is possible had better be able to draw a blueprint describing the thing’s composition and write down the equations governing its function. Otherwise I call bullshit. (Yes, I’m aware I am calling bullshit on a number of philosophers, here.)