Dweomite

• Dweomite 5 May 2026 21:47 UTC

  16 points

  0

  in reply to: Matthew Barnett’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  I would describe a critical try as one where the act of trying is likely to prevent further attempts. Launching an ASI is a critical try because the ASI itself could likely stop you from launching more ASIs later on (e.g. by killing you).

  If it’s possible to send out missions to intercept the asteroid before it arrives, then it seems to me that the asteroid is better understood as a time limit than as a critical try. You could set the parameters of the asteroid scenario in such a way that you have time for exactly one try, but you could also set the parameters so that you have time to send up a mission to deflect the asteroid, observe its results, and then make a second try before the asteroid arrives. You could also set the parameters such that you have time for zero tries! The key consideration is how fast you can work vs how much time you have.

  Contrariwise, if you assume that you are stopping the asteroid with a shield that is close to the earth, such that no matter how fast you build the shield you have to wait for the asteroid to arrive before you can see how well it works, then I’d call that a critical try, because the part of the plan where you wait for the asteroid to arrive severely depletes a critical resource (time) and makes that resource unavailable for later attempts. (Note similarity to the Maginot Line.)

  By similar reasoning, I’d say your #2 (global warming) is also more of a time limit, but your #3 (creating a new type of human that potentially kills you) is a critical try (though compared to launching an ASI, it’s more likely to get a middle-ground outcome).

• Dweomite 22 Apr 2026 7:30 UTC

  2 points

  0

  in reply to: polytope’s comment on: An­noy­ingly Prin­ci­pled Peo­ple, and what be­falls them

  Yes, that is the sort of example I meant. Though of course this particular example does not prove that the game of Catan, in particular, has situations like this.

  Based on his other reply, I expect James would want to point out that there is an equivalent equilibrium where player A, instead of saying “button N is blue”, says “either button N is blue or no button is”, which produces the same outcome without technically lying.

  I’m coming to think that there should be some other distinction we can draw that rhymes with the truthful/​lying distinction but that talks about consequences instead of semantics, and therefore can’t be dodged by relabeling the signals. Still thinking about it.

• Dweomite 22 Apr 2026 4:20 UTC

  2 points

  0

  in reply to: James Camacho’s comment on: An­noy­ingly Prin­ci­pled Peo­ple, and what be­falls them

  In principle, IF the norms are more important to you than to everyone else combined, then there should be some amount you can pay them that is higher than how much they care about the norms but lower than how much you care about them.

  (In practice, finding that amount may be hard, and treating it too much like a transaction may have friendship-corroding effects.)

• Dweomite 22 Apr 2026 4:15 UTC

  2 points

  0

  in reply to: James Camacho’s comment on: An­noy­ingly Prin­ci­pled Peo­ple, and what be­falls them

  Your final sentence clarified some things for me:

  • In that sentence, you are not arguing that the lie is no better than silence, you are arguing that it is no better than some truthful message. (This is technically still a falsification of my previously-stated interpretation of your claim.)

  • This argument is based on the assumption that the other players already know all circumstances under which you would transmit this message, so there’s no harm in admitting them.

  I now realize that if all players have perfect knowledge of the exact conditions under which you would transmit some message, then the actual informational payload of every message is that those conditions are true. (Even with a randomized strategy, you can just interpret the RNG output as part of the conditions.) You might as well literally say “message #27”. Classifying the message itself as truth or lie becomes academic, because no one is expected or intended to pay attention to its face-value claim, and in fact there’s no reason for it to make a face-value claim at all. (Under this very strong assumption.)

  So if we’re going to assume players have perfect knowledge of each others’ strategies (including what messages they send under what circumstances), I no longer think it makes sense to distinguish “true” and “false” messages.

  I note that “common knowledge that all players are perfectly rational” does not (I think) logically entail perfect knowledge of everyone’s strategy, since a game can have more than one Nash equilibrium. So technically neither of us stated “perfect knowledge of everyone’s strategy” as an assumption in the first place, though I admit I sort of hand-waved towards it when I talked about what players would infer from your failure to say something.

  I still think that if we don’t assume “perfect knowledge of everyone’s strategy” then lying is potentially beneficial.

  ---

  Given that clarification, I’m not sure if your numbered chain of reasoning is a crux for either of us, but for the record I found that chain extremely confusing to read, I think step 3 is invalid, and your final paragraph (after the numbered list) was the only part of the comment I found helpful.

  In step 3, you seem to be trying to treat the groups and as if they were each a single player so that you can apply the conclusions from two-player games, but I don’t think that’s valid. The two-player result was based on an implicit assumption that transmitting a message from Beth to Adam cannot have any effect on the game except through Adam’s reaction, but that’s not true here because isn’t a unified agent, so transmitting can change the game (by affecting other members of ) even if refuses to react to it. does not get a veto on changing the game, like Adam does. Chris does not need to be listening in order for Adam and Beth to strike a mutually-beneficial deal at his expense.

  So the inference that as a group cannot be profiting is invalid.

  (Also note that your claim proves too much: If this were accepted, you haven’t proven that false messages are useless, you’ve proven that all messages are useless.)

• Dweomite 21 Apr 2026 21:57 UTC

  13 points

  8

  in reply to: James Camacho’s comment on: An­noy­ingly Prin­ci­pled Peo­ple, and what be­falls them

  Been thinking more about this claim:

  Also, with rational agents silence is just as good as dishonesty.

  I don’t think this claim particularly matters to the thrust of your post, since I think we agree that you’re not playing with perfectly rational agents, but I’m interested in the claim as a matter of game theory.

  To be clear, I’m interpreting this as saying something at least as strong as: “In a game of Catan where there is common knowledge that all players are perfectly rational, speaking a falsehood is never more advantageous for the speaker than remaining silent.”

  After pondering this for about 20 minutes, I’m pretty convinced the claim is false, and I suspect you are over-generalizing from two-player games.

  If Adam and Beth are playing a two-player zero-sum game, and Adam knows that Beth is perfectly rational, then:

  1. If Beth reacts in any way to anything Adam says, then that reaction must be beneficial to Beth (since she is assumed perfectly rational), which means it must be harmful to Adam (because the game is zero-sum), which means Adam shouldn’t have said it.

  2. By similar reasoning, if Beth says anything (that’s not required by the rules), then the fact that she said it can’t be harmful to herself, which means it can’t be beneficial to Adam, which means whatever Adam does in response won’t be (predictably) better than what he would have done anyway.

  Therefore, Adam can safely adopt a policy of never saying anything and ignoring whatever Beth says, and this will be no worse than any other policy.

  But Catan is played with at least 3 players. The game as a whole is zero-sum, but it’s possible for an action to benefit both Adam and Beth at the same time, provided it harms Chris.^[1]

  In a non-zero-sum negotiation, it is sometimes helpful to share information in order to coordinate on a mutually-beneficial action. So silence is not, in general, a global optimum.

  But if there are situations where you would share some information if it were true, and the other player is aware of this, then silence becomes a tacit admission that it’s not true. So it might become necessary to lie in order to avoid passively leaking secrets.

  The lie will only be believable if it’s a claim you would have made if it were true, which sharply limits what lies you can tell. But it does not, in general, limit it to the empty set.

  This is not a proof, since I have not constructed an example game position where I can mathematically demonstrate that all of the relevant properties apply at the same time. It is conceivable there’s some reason that hasn’t occurred to me that they can’t all apply at the same time. But I have no candidates for what such a reason would be, and my brief Internet searches have failed to turn up any known result that matches the original claim.

  Do you think I’ve missed something?

  1. ^

     I’m not sure if this is known art, but I’ve found it helpful to think of zero-sum-ness as applying to a set of players rather than to a game. In a 3-player Catan game with Adam, Beth, and Chris, the set (Adam, Beth, Chris) is zero-sum, but the set (Adam, Beth) is non-zero-sum.
     Note that any non-zero-sum game can be converted to a strategically-equivalent zero-sum game by adding a dummy player whose score is the negative sum of all other players’ scores (or vice versa, by adding a dummy player whose score isn’t that), so it cannot be strategically important whether “the whole game” is zero-sum if we haven’t changed the zero-sum-ness of any particular subset of players.

• Dweomite 21 Apr 2026 6:56 UTC

  9 points

  6

  in reply to: James Camacho’s comment on: An­noy­ingly Prin­ci­pled Peo­ple, and what be­falls them

  Have you talked explicitly with them about the norms you’d like to have? I, for one, would not have assumed that “don’t try to manipulate other players to your own advantage” would be an expected norm, but would probably be willing to go along with it if the group asked me to.

  You also might consider offering to play with a handicap, so that they don’t feel that they need to target you to prevent you from winning too often.

  As a rule of thumb, I strongly approve of play groups mutually agreeing on whatever rules and norms work best for them. But I also think that trying to win (within the rules) is a pretty good default norm, and shouldn’t be interpreted as a defection if you haven’t agreed on something else. I don’t think “having honest conversations” is the primary value proposition that games offer to most gamers, and in fact I can think of several popular games with dynamics that preclude it.

  I do notice that you seem quite confident that this is harming your enjoyment more than it’s helping anyone else’s, and this seems...plausible, but not self-evident to me, based on the information provided. Some people really like politicking in games. It’s also the sort of thing you’d be tempted to believe even if it weren’t true, which is cause for epistemic caution.

  Supposing it’s true that this is more important to you than to everyone else combined, then I think they probably ought to be willing to negotiate to follow your norms, but that you should expect to give them something else in return (even if it’s just owing them one). Try to strike a deal that’s a positive for every individual, not merely positive-sum. You shouldn’t be able to demand people accommodate you just by being a utility monster. (Though you absolutely should be allowed to stop playing, if that’s your BATNA...and if they care more about having you play with them than they do about the norms, then playing with them could perhaps be the payment for changing the norms.)

• Dweomite 17 Apr 2026 7:02 UTC

  10 points

  6

  in reply to: Gretta Duleba’s comment on: Spe­cial­iza­tion is a Driver of Nat­u­ral Ontology

  • Identical twins share the same genome, but are usually considered separate objects for most purposes.

  • A genetic chimera has different genomes in different parts of its body, but is usually considered a single object for most purposes. (Often, you can’t tell the organism is a chimera on a casual inspection.)

  I submit that “same genome” often coincides with the natural object boundary, but isn’t usually a good criterion for the boundary. The common genome is not a significant part of what makes a squirrel a good object.

  I think the thing we usually care about is something more like “it acts like a single agent” or “the parts are approximately aligned towards the same goal”. (Notice these criteria suggest that when an organization acts in concert, we care more about the boundaries of the organization than the boundaries of the individuals within it; I think I endorse that implication.)

  Interestingly, according to Wikipedia there doesn’t seem to be an accepted scientific definition of “organism”.

• Dweomite 10 Apr 2026 23:01 UTC

  7 points

  0

  in reply to: cousin_it’s comment on: Socrates is Mortal

  Based on this and your other comments in this thread, I suspect you’re mixing up questions of

  1. What’s good?

  2. How do we know what’s good?

  3. How should we treat other people who disagree with us about what’s good?

  It’s possible to think someone is baking bread wrong without thinking that you should use violence to force them to do it differently. It’s possible to think that bakers should be allowed to pick their own baking methods without thinking that all methods produce equally tasty bread.

  Civilization typically uses many different levels of coercion for different sorts of rules. Different offenses might get you jail, or fines, or social censure. This difference isn’t because some of those offenses are Wrong and others are Not Wrong; it’s because creating effective policies of enforced cooperation is more complicated than just asking whether some action is Wrong. (I think this essay from Scott Alexander significantly improved my thinking on this topic.)

  I think your hypothetical baker has a good and sufficient answer to your hypothetical philosopher, which is that they actually make good bread. I broadly agree that you don’t need a theoretical understanding of why your practices are good if you have some other valid reason for thinking they’re good.

  But while the bakers with the best recipes won’t necessarily have good theoretical explanations, that doesn’t mean that there are no bad recipes, or that you can identify good recipes by sheer intuition. You do still need actual entangled evidence of some kind to reach accurate conclusions.

  If we accept (as you wrote in another comment) that “success in convincing is the sole criterion”, then the best conclusion is one enforced by a mind control ray. I think that’s nonsense. In real life, people frequently believe things for reasons that are not much correlated with accuracy.

  Furthermore, using this as the defense of any specific conclusion is circular. We, who are questioning the conclusion, are included in the group of “people”, and we have not been convinced. Insofar as your proposed system works at all, it only works because you are depending on “people” to be more likely to be convinced of correct conclusions than incorrect conclusions. If everyone who might object to the decision allowed themselves to be convinced merely because others are convinced, we’d be removing the only element of this system that makes it better than chance.

• Dweomite 15 Mar 2026 21:32 UTC

  8 points

  0

  on: On The In­de­pen­dence Axiom

  I think you do a good job of arguing (in the earlier part of the article) that it is logically possible to drop the independence axiom without being money-pumped by giving up logical consequentialism but keeping dynamic consistency. However, I think you do a poor job of arguing (in the later parts) that we should give up consequentialism.

  You examine 3 in-depth examples to try to show that we’d be fine if we dropped independence: ergodicity economics, the Allais Paradox, and the Ellsburg Paradox. In all 3 cases, it think your argument is missing a critical step that is required for its validity.

  1.

  In the section on ergodicity economics, you claim ergodicity follows resolute choice because it forms a plan based on the entire decision tree and then sticks to that plan. But this isn’t sufficient to carry your point, because agents that obey the independence axiom can also be described as sticking to their original plan. (In fact, any agent with dynamic consistency can be described this way, and you agreed we need dynamic consistency.)

  What you’d need to show in order to carry your point is that ergodicity violates consequentialism. For example, you could show this by constructing an example where a local re-evaluation would deviate from the original plan, but ergodicity follows the original plan anyway. Without showing that, this example fails to support your case.

  2.

  In the section on the Allais Paradox, you give the following reasoning for why the common human answer is rational:

  This is precisely the point we made with the example in the introduction to section 3. If the common component C is a large safety net, you can afford to take more risk on the remaining branch. If C is negligible, you should be more conservative. Your preference between A and B should depend on what else is in the package, because you are one agent facing the total distribution, not a collection of independent sub-agents each evaluating one branch in isolation.

  But this reasoning seems to be exactly backwards from the actual result: When component C provides a safety net of $1M, humans choose the lower-risk option A, but when component C provides nothing, humans choose the higher-risk option B. Your argument in this paragraph undermines, rather than supports, the rationality of the choice you are defending.

  And aside from this one backwards paragraph, you don’t seem to offer any basis at all for how the context ought to change the answer. You have several paragraphs of philosophical hand-waving about how it is good and appropriate that it should, but don’t appear to offer anything like an algorithm saying how we should take it into account. Without a model predicting the preference for A over B, you fail to win any Bayes points.

  Nothing in this section sounds like a logical reason to consider the common human choice in the Allais Paradox more rational than I previously did.

  Sidenote: Empirical Money Pumps?

  This discussion also suggests the question: Can you actually, in real life, use the Allais Paradox to money-pump humans? If you can, then the behavior of humans does not provide evidence of the rationality of their choices in this scenario, regardless of any theoretical arguments about how we could avoid money pumps while keeping this preference. My brief Google search failed to immediately turn up any experiments involving actual money pumps, but I haven’t done a careful literature review.

  Sidenote: Can the Allais Paradox result be justified in other ways?

  There’s two defenses for this that I somewhat credit:

  A.

  Eliminating a possible outcome makes it cognitively cheaper to plan for what happens after the lottery, because you don’t need to consider as many distinct cases.

  Notice this reasoning only applies if there is an “after”, which is usually true in real life but usually false in abstract formal examples.

  B.

  Suppose you are living among a population of similar agents that compete for resources, and all of the other agents get to make a similar choice between lotteries. Then, the outcome where you get nothing is always the same in terms of absolute resources, but not in terms of relative resources when comparing to other people.

  If you choose between a 1% chance and a 0% chance of getting nothing, then the few agents who end up with nothing will be out-competed by almost everyone around them. They will lose approximately all competitions and will be the obvious choice for predators to target.

  If you choose between a 90% chance and a 89% chance of getting nothing, then agents who win millions will still out-compete the ones who get nothing, but they’ll have a harder time monopolizing all opportunities because there won’t be as many winners. Many of the “losers” will still have a decent relative standing.

  This reasoning doesn’t apply if you somehow know this lottery is a special one-time opportunity for you only, but it seems plausible that our instincts evolved mostly to deal with non-unique opportunities.

  However, notice that these two reasons justify different things. Reason A justified zero-risk bias, i.e. paying a premium to reduce a risk to ~zero; it has a sharp change in your preferences at a specific probability. Contrariwise, reason B would remain nearly as strong if we changed “1% or 0%” to “2% or 1%”.

  3.

  In the section on the Ellsberg Paradox, I think you make some clever points about why the standard human answer might be rational, but I don’t see how any part of this section ties into logical consequentialism or the violation thereof. For example, you have not explained how a money pump could be constructed based on this scenario.

  4.

  The argument in favor of logical consequentialism is obvious: If you violate it, you are leaving money on the table. (Violating it implies that you are making a choice that satisfies your own preferences less than another choice you could have made in the current circumstances.)

  In fact, this is essentially the same reason that we think that vulnerability to money pumps is bad (you end up with less money than you predictably could have). So it seems pretty weird to argue that we need to keep all the axioms that prevent money pumps but it’s somehow ok to drop consequentialism. I’m not sure what set of assumptions would validly lead to that combination of conclusions.

• Dweomite 22 Jan 2026 8:17 UTC

  2 points

  0

  in reply to: jimmy’s comment on: Why Mo­ti­vated Rea­son­ing?

  Sure, give me meta-level feedback.

• Dweomite 19 Jan 2026 0:16 UTC

  2 points

  0

  in reply to: jimmy’s comment on: Why Mo­ti­vated Rea­son­ing?

  If it has an integral gain, it will notice this and try to add more and more heat until it stops being wrong. If it can’t, it’s going to keep asking for more and more output, and keep expecting that this time it’ll get there. And because it lacks the control authority to do it, it will keep being wrong, and maybe damage its heating element by asking more than they can safely do. Sound familiar yet?

  From tone and context, I am guessing that you intend for this to sound like motivated reasoning, even though it doesn’t particularly remind me of motivated reasoning. (I am annoyed that you are forcing me to guess what your intended point is.)

  I think the key characteristic of motivated reasoning is that you ignore some knowledge or model that you would ordinarily employ while under less pressure. If you stay up late playing Civ because you simply never had a model saying that you need a certain amount of sleep in order to feel rested, then that’s not motivated reasoning, it’s just ignorance. It only counts as motivated reasoning if you, yourself would ordinarily reason that you need a certain amount of sleep in order to feel rested, but you are temporarily suspending that ordinary reasoning because you dislike its current consequences.

  (And I think this is how most people use the term.)

  So, imagine a scenario where you need 100J to reach your desired temp but your heating element can only safely output 50J.

  If you were to choose to intentionally output only 50J, while predicting that this would somehow reach the desired temperature (contrary to the model you regularly employ in more tractable situations), then I would consider that a central example of motivated reasoning. But your model does not seem to me to explain how this strategy arises.

  Rather, you seem to be describing a reaction where you try to output 100J, meaning you are choosing an action that is actually powerful enough to accomplish your goal, but which will have undesirable side-effects. This strikes me as a different failure mode, which I might describe as “tunnel vision” or “obsession”.

  I suppose if your heating element is in fact incapable of outputting 100J (even if you allow side-effects), and you are aware of this limitation, and you choose to ask for 100J anyway, while expecting this to somehow generate 100J (directly contra the knowledge we just assumed you have), then that would count as motivated reasoning. But I don’t think your analogy is capable of representing a scenario like this, because you are inferring the controller’s “expectations” purely from its actions, and this type of inference doesn’t allow you to distinguish between “the controller is unaware that its heating element can’t output 100J” from “the controller is aware, but choosing to pretend otherwise”. (At least, not without greatly complicating the example and considering controllers with incoherent strategies.)

  Meta-level feedback: I feel like your very long comment has wasted a lot of my time in order to show off your mastery of your own field in ways that weren’t important to the conversation; e.g. the stuff about needing to react faster than the thermometer never went anywhere that I could see, and I think your 5-paragraph clarification that you are interpreting the controller’s actions as implied predictions could have been condensed to about 3 sentences. If your comments continue to give me similar feelings, then I will stop reading them.

• Dweomite 16 Jan 2026 19:50 UTC

  2 points

  0

  in reply to: jimmy’s comment on: Why Mo­ti­vated Rea­son­ing?

  At some point, a temperature control system needs to take actions to control the temperature. Choosing the correct action depends on responding to what the temperature actually is, not what you want it to be, or what you expect it to be after you take the (not-yet-determined) correct action.

  If you are picking your action based on predictions, you need to make conditional predictions based on different actions you might take, so that you can pick the action whose conditional prediction is closer to the target. And this means your conditional predictions can’t all be “it will be the target temperature”, because that wouldn’t let you differentiate good actions from bad actions.

  It is possible to build an effective temperature control system that doesn’t involve predictions at all; you can precompute a strategy (like “turn heater on below X temp, turn it off above Y temp”) and program the control system to execute that strategy without it understanding how the strategy was generated, and in that case it might not have models or make predictions at all. But if you were going to rely on predictions to pick the correct action, it would be necessary to make some (conditional) predictions that are not simply “I will succeed”.

• Dweomite 15 Jan 2026 1:17 UTC

  10 points

  1

  on: Why Mo­ti­vated Rea­son­ing?

  Your explanation about the short-term planner optimizing against the long-term planner seems to suggest we should only see motivated reasoning in cases where there is a short-term reward for it.

  It seems to me that motivated reasoning also occurs in cases like gamblers thinking their next lottery ticket has positive expected value, or competitors overestimating their chances of winning a competition, where there doesn’t appear to be a short-term benefit (unless the belief itself somehow counts as a benefit). Do you posit a different mechanism for these cases?

  I’ve been thinking for a while that motivated reasoning sort of rhymes with reward hacking, and might arise any time you have a generator-part Goodharting an evaluator-part. Your short-term and long-term planners might be considered one example of this pattern?

  I’ve also wondered if children covering their eyes when they get scared might be an example of the same sort of reward hacking (instead of eliminating the danger, they just eliminate the warning signal from the danger-detecting part of themselves by denying it input).

• Dweomite 5 Jan 2026 3:33 UTC

  2 points

  0

  in reply to: jbash’s comment on: The Weird­ness of Dat­ing/​Mat­ing: Deep Non­con­sent Preference

  … except that you have a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations. Or at least I think both of those are true of most people.

  It seems pretty important to me to distinguish between “heuristic X is worse than its inverse” and “heuristic X is better than its inverse, but less good than you think it is”.

  Your top-level comment seemed to me like it was saying that a given simple explanation is less likely to be true than a given complex explanation. Here, you seem to me like you’re saying that simple explanation is more likely to be true, but people have a preference for them that is stronger than the actual effect, and so you want to push people back to having a preference that is weaker but still in the original direction.

• Dweomite 9 Dec 2025 0:56 UTC

  2 points

  0

  in reply to: J Bostock’s comment on: The Most Com­mon Bad Ar­gu­ment In Th­ese Parts

  “Possible” is a subtle word that means different things in different contexts. For example, if I say “it is possible that Angelica attended the concert last Saturday,” that (probably) means possible relative to my own knowledge, and is not intended to be a claim about whether or not you possess knowledge that would rule it out.

  If someone says “I can(not) imagine it, therefore it’s (not) possible”, I think that is valid IF they mean “possible relative to my understanding”, i.e. “I can(not) think of an obstacle that I don’t see any way to overcome”.

  (Note that “I cannot think of a way of doing it that I believe would work” is a weaker claim, and should not be regarded as proof that the thing is impossible even just relative to your own knowledge.)

  If that is what they mean, then I think the way to move forward is for the person who imagines it impossible to point out an obstacle that seems insurmountable to them, and then the person who imagines it possible to explain how they imagine solving it, and repeat.

  If someone is trying to claim that their (in)ability to imagine something means that the laws of the universe (dis)allow it, then I think the person imagining it is impossible had better be able to point out a specific conflict between the proposal and known law, and the person imagining it is possible had better be able to draw a blueprint describing the thing’s composition and write down the equations governing its function. Otherwise I call bullshit. (Yes, I’m aware I am calling bullshit on a number of philosophers, here.)

• Dweomite 8 Dec 2025 22:55 UTC

  4 points

  0

  in reply to: plex’s comment on: The Most Com­mon Bad Ar­gu­ment In Th­ese Parts

  I interpreted the name as meaning “performed free association until the faculty of free association was exhausted”. It is, of course, very important that exhausting the faculty does not guarantee that you have exhausted the possibility space.

• Dweomite 15 Nov 2025 1:51 UTC

  8 points

  0

  on: Please, Don’t Roll Your Own Metaethics

  Alas, unlike in cryptography, it’s rarely possible to come up with “clean attacks” that clearly show that a philosophical idea is wrong or broken.

  I think the state of philosophy is much worse than that. On my model, most philosophers don’t even know what “clean attacks” are, and will not be impressed if you show them one.

  Example: Once in a philosophy class I took in college, we learned about a philosophical argument that there are no abstract ideas. We read an essay where it was claimed that if you try to imagine an abstract idea (say, the concept of a dog), and then pay close attention to what you are imagining, you will find you are actually imagining some particular example of a dog, not an abstraction. The essay went on to say that people can have “general” ideas where that example stands for a group of related objects rather than just for a single dog that exactly matches it, but that true “abstract” ideas don’t exist.^[1]

  After we learned about this, I approached the professor and said: This doesn’t work for the idea of abstract ideas. If you apply the same explanation, it would say: “Aha, you think you’re thinking of abstract ideas in the abstract, but you’re not! You’re actually thinking of some particular example of an abstract idea!” But if I’m thinking of a particular example, then there must be at least one example to think of, right? So that would prove there is at least one member of the class of abstract ideas (whatever “abstract ideas” means to me, inside my own head). Conversely, if I’m not thinking of an example, then the paper’s proposed explanation is wrong for the idea of abstract ideas itself. So either way, there must be at least one idea that isn’t correctly explained by the paper.

  The professor did not care about this argument. He shrugged and brushed it off. He did not express agreement, he did not express a reason for disagreement, he was not interested in discussing it, and he did not encourage me to continue thinking about the class material.

  On my model, the STEM fields usually have faith in their own ideas, in a way where they actually believe those ideas are entangled with the Great Web. They expect ideas to have logical implications, and expect the implications of true ideas to be true. They expect to be able to build machines in real life and have those machines actually work. It’s something like taking ideas seriously, and something like taking logic seriously, and taking the concept of truth seriously, and seriously believing that we can learn truth if we work hard. I’m not sure if I’ve named it correctly, but I do think there’s a certain mental motion of genuine truth-seeking that is critical to the health of these fields and that is much less common in many other fields.

  Also on my model, the field of philosophy has even less of this kind of faith than most fields. Many philosophers think they have it, but actually they mostly have the kind of faith where your subconscious mind chooses to make your conscious mind believe a thing for non-epistemic reasons (like it being high-status, or convenient for you). And thus, much of philosophy (though not quite all of it) is more like culture war than truth-seeking (both among amateurs and among academics).

  I think if I had made an analogous argument in any of my STEM classes, the professor would have at least taken it seriously. If they didn’t believe the conclusion but also couldn’t point out a specific invalid step, that would have bothered them.

  I suspect my philosophy professor tagged my argument as being from the genre of math, rather than the genre of philosophy, then concluded he would not lose status for ignoring it.

  1. ^

     I think this paper was clumsily pointing to a true and useful insight about how human minds naturally tend to use categories, which is that those categories are, by default, more like fuzzy bubbles around central examples than they are like formal definitions. I suspect the author then over-focused on visual imagination, checked a couple of examples, and extrapolated irresponsibly to arrive at a conclusion that I hope is obviously-false to most people with STEM backgrounds.

• Dweomite 13 Nov 2025 5:53 UTC

  6 points

  0

  on: Hu­man Values ≠ Goodness

  An awful lot of people, probably a majority of the population, sure do feel deep yearning to either inflict or receive pain, to take total control over another or give total control to another, to take or be taken by force, to abandon propriety and just be a total slut, to give or receive humiliation, etc.

  This is rather tangential to the main thrust of the post, but a couple of people used a react to request a citation for this claim.

  One noteworthy source is Aella’s surveys on fetish popularity and tabooness. Here is an older one that gives the % of people reporting interest, and here is a newer one showing the average amount of reported interest on a scale from 0 (none) to 5 (extreme), both with tens of thousands of respondents.

  Very approximate numbers that I’m informally reading off the graphs:

  • Giving pain: 30% of people interested (first graph), ²⁄₅ average interest (second graph)

  • Receiving pain: 35% and ²⁄₅

  • Being dominant: 30% and ³⁄₅

  • Being submissive: 40% and ³⁄₅

  • Rapeplay: >10% giving, 20% receiving, the second graph combines these at ²⁄₅

  • Slut Humiliation (first graph): 25%

  • Humiliation (second graph): ²⁄₅

  Note that a ³⁄₅ average interest could mean either that 60% of people are extremely into it or that nearly everyone is moderately into it (or anything in between). Which seems to imply the survey used in the more recent graph has significantly kinkier answers overall, unless I’m misunderstanding something. (I’m fairly certain that people with zero interest ARE being included in the average, because several other fetishes have average interest below 1, which should be impossible if not.)

  If we believe this data, it seems pretty safe to guess that a majority of people are into at least one of these things (unless there is near-total overlap between them). The claim that a majority “feel a deep yearning” is not strongly supported but seems plausible.

  (I was previously aware that BDSM interest was pretty common for an extremely silly reason: I saw some people arguing about whether or not Eliezer Yudkowsky was secretly the author of The Erogamer, one of them cited the presence of BDSM in the story as evidence in favor, and I wanted to know the base rate to determine how to weigh that evidence.

  I made an off-the-cuff guess of “between 1% and 10%” and then did a Google search with only mild hope that this statistic would be available. I wasn’t able today to re-find the pages I found then, but according to my recollection, my first search result was a page describing a survey of ~1k people claiming a ~75% rate of interest in BDSM, and my second search result was a page describing a survey of ~10k people claiming ~40% had participated in some form of BDSM and an additional ~40% were interested in trying it. I was also surprised to read (on the second page) that submission was more popular than dominance, masochism was more popular than sadism, and masochism remained more popular than sadism even if you only looked at males. Also, bisexuality was reportedly something like 5x higher within the BDSM-interested group than outside of it.)

• Dweomite 12 Nov 2025 23:39 UTC

  6 points

  0

  in reply to: Nina Panickssery’s comment on: Hu­man Values ≠ Goodness

  If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.

  I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/​mental state”, then why would the things humans actually value necessarily be the same as Goodness?

• Dweomite 12 Nov 2025 23:27 UTC

  3 points

  0

  in reply to: Wei Dai’s comment on: Hu­man Values ≠ Goodness

  What would you want such a disclaimer or hint to look like?

  (I am concerned that if a post says something like “this post is aimed at low-level people who don’t yet have a coherent foundational understanding of goodness and values” then the set of people who actually continue reading will not be very well correlated with the set of people we’d like to have continue reading.)