Humor? I present a concrete, executable plan to build CEV, save humanity, and get tagged as Humor?
Yeah so what are the norms on that? It seems like putting on a character, dialing up the chaos, and pretending without a shred of humility that I’m Douglas Adams has gotten the comments section to engage with the ideas I’m interested in without any hints of isolated demand for rigor or questions about my lack of citations. (I did in fact flub the date and location of cat domestication: it’s 7500 BC in mesopotamia.)
It definitely works as a rhetorical strategy! I think it’s pretty safe for the reader’s behavior—no one is going to break ground on 100 mecha-silos for this post, because they know I’m joking. It is, by default damaging to community epistemology. The issue is that some of the claims in the post are just true, some are just false (don’t give me the red button,) but most are on the border, and it’s really easy for each reader to assume that they agree with me on what’s true and what’s a joke, without necessarily agreeing with me or each other. To that end, a list of the claims which I sincerely defend: this subtree of the comment section is for roasting me for these dumb takes
Evolutionary pressure is an intelligence we need to model, and pretending that it doesn’t move with intentionality makes the world unnecessarily hard to understand. (Intentionality, not sentience.) In particular, looking at the outcomes achieved in ecosystems, the “shoggoths” of various species are incredibly adept at making FDT style trades with each other, despite appearing completely myopic. I don’t understand this and want to.
I observe the human relationship with cats is in the attractor of legitimate CEV, and has been in this attractor for all of recorded history: our abuses towards cats have been skill issues on our end. With high confidence, we are going to do right by cats or die trying, and fundamentally as we get better at doing things, we’re going to get better at not dying trying. This means that CEV actually has an attractor around it, which is not a claim I have believed before, or would want to make without thousands of years of evidence across multiple total changes of moral structure, epistemology, and capability.
We did actually learn a utility function from data and then satisfice it in a reasonable way- no one is building rooms full of cats on heroin.
The alignment of humans to cats wasn’t an accident per se: it was instrumentally useful to the cat shoggoth for a period of several hundred to a thousand years, in order to somewhat increase the cat population in one civilization. So, the cat shoggoth did it. Consider this reasoning: “I wanted to create several thousand to a million more cats. So, I permanently solved a deep game-theory/philosophy problem that appears completely intractably to hundreds of years of human analytic philosophy.” This reasoning seems completely in character for evolution- stunningly brilliant engineer, no concept of proportionality between difficulty of task and concrete results, no requirement to do any task in the most efficient way. I absolutely don’t claim that the cat shoggoth cared about individual well being- it just randomly came upon a chance to point human brain power at cat well being, measured that exploiting this opening made cats progressively better at reproducing, and then correctly implemented alignment in order to realize this fitness gain. I’m open to the possibility that this sequence of events was phenomenally unlikely.
Claims I don’t defend: The plan of re-domesticating cats via AI grain warehouses wouldn’t work as written. Instantiating a neo-Bastet smart enough to be aligned to cats in the way that we are, using LLM components, and giving it any power at all would be wildly unsafe, and also is likely impossible. The big problem is that I don’t have any proposal for how to measure whether a higher intelligence is is aligned to a lower intelligence other than just letting the higher intelligence run roughshod over a world and observing what it does, so checking that an AI is aligned to cats is not any easier than checking if an AI is aligned to humans.
(a sharp distinction from Sniffles the teacup poodle. I don’t care if you think you’re happy, this would not please the prowling wolves of the stone age.)
I have issues with this: I don’t think you can claim that wildcats of the stone age would be pleased with what we’ve done to domestic cats either, sticking them in tiny territories where they cannot roam, kingdoms of a cage. I’m not sure using human judgement in this matter is very useful as we don’t have a good concept of what other species value.
I don’t think evolutionary pressure is an intelligence; it’s in the name. It’s a pressure, like air pressure or water pressure, not an intelligence. There is no agency. The end results can be marvelous all the same. Evolution appears to make FDT-style trades, but is actually completely myopic. It’s survival of the survivors. If you’re dead, we don’t see you.
Also sociality in spiders has evolved independently at least twenty times, and keeps going extinct. It’s probably an evolutionary dead end due to inbreeding [1]. Evolution is completely myopic.
no one is building rooms full of cats on heroin.
We do, however, build rooms full of cats and catnip.
I don’t think evolutionary pressure is an intelligence; it’s in the name. It’s a pressure, like air pressure or water pressure, not an intelligence. There is no agency. The end results can be marvelous all the same. Evolution appears to make FDT-style trades, but is actually completely myopic. It’s survival of the survivors. If you’re dead, we don’t see you.
I don’t think we are materially disagreeing here, just working with very slight differences in definition of intelligence. I see a strong analogy to the debate between whether LLMs think or LLMs just predict the next token. I think that your claims that “There is no agency” and evolution is “completely myopic” are true for some reasonable definitions of agency and myopia, as long as you don’t make any arguments like “There is no agency / complete myopia, therefore evolution can’t X.” You aren’t making any such arguments, hence the lack of material disagreement.
On the subject of cats being deprived of roaming and hopped up on catnip, yeah I would not love that either, but the contrast to the outcomes for cows or chickens is big. Getting alignment close enough that the result is preferable to going extinct is a high bar. To reiterate, we could also just stop, but that would be ludicrous.
I don’t think you can claim that wildcats of the stone age would be pleased with what we’ve done to domestic cats either, sticking them in tiny territories where they cannot roam, kingdoms of a cage. I’m not sure using human judgement in this matter is very useful as we don’t have a good concept of what other species value.
Just one factor, but the life expectancy of domestic dogs and cats is generally higher than their wild progenitors. I agree we can’t know for sure, but I would guess this with limitless food and good healthcare, and less worry about being attacked at night would mean the subjective wellbeing of domesticated cats and dogs is higher than the wild ones, despite less freedom.
Humor? I present a concrete, executable plan to build CEV, save humanity, and get tagged as Humor?
Yeah so what are the norms on that? It seems like putting on a character, dialing up the chaos, and pretending without a shred of humility that I’m Douglas Adams has gotten the comments section to engage with the ideas I’m interested in without any hints of isolated demand for rigor or questions about my lack of citations. (I did in fact flub the date and location of cat domestication: it’s 7500 BC in mesopotamia.)
It definitely works as a rhetorical strategy! I think it’s pretty safe for the reader’s behavior—no one is going to break ground on 100 mecha-silos for this post, because they know I’m joking. It is, by default damaging to community epistemology. The issue is that some of the claims in the post are just true, some are just false (don’t give me the red button,) but most are on the border, and it’s really easy for each reader to assume that they agree with me on what’s true and what’s a joke, without necessarily agreeing with me or each other. To that end, a list of the claims which I sincerely defend: this subtree of the comment section is for roasting me for these dumb takes
Evolutionary pressure is an intelligence we need to model, and pretending that it doesn’t move with intentionality makes the world unnecessarily hard to understand. (Intentionality, not sentience.) In particular, looking at the outcomes achieved in ecosystems, the “shoggoths” of various species are incredibly adept at making FDT style trades with each other, despite appearing completely myopic. I don’t understand this and want to.
I observe the human relationship with cats is in the attractor of legitimate CEV, and has been in this attractor for all of recorded history: our abuses towards cats have been skill issues on our end. With high confidence, we are going to do right by cats or die trying, and fundamentally as we get better at doing things, we’re going to get better at not dying trying. This means that CEV actually has an attractor around it, which is not a claim I have believed before, or would want to make without thousands of years of evidence across multiple total changes of moral structure, epistemology, and capability.
We did actually learn a utility function from data and then satisfice it in a reasonable way- no one is building rooms full of cats on heroin.
The alignment of humans to cats wasn’t an accident per se: it was instrumentally useful to the cat shoggoth for a period of several hundred to a thousand years, in order to somewhat increase the cat population in one civilization. So, the cat shoggoth did it. Consider this reasoning: “I wanted to create several thousand to a million more cats. So, I permanently solved a deep game-theory/philosophy problem that appears completely intractably to hundreds of years of human analytic philosophy.” This reasoning seems completely in character for evolution- stunningly brilliant engineer, no concept of proportionality between difficulty of task and concrete results, no requirement to do any task in the most efficient way. I absolutely don’t claim that the cat shoggoth cared about individual well being- it just randomly came upon a chance to point human brain power at cat well being, measured that exploiting this opening made cats progressively better at reproducing, and then correctly implemented alignment in order to realize this fitness gain. I’m open to the possibility that this sequence of events was phenomenally unlikely.
Claims I don’t defend: The plan of re-domesticating cats via AI grain warehouses wouldn’t work as written. Instantiating a neo-Bastet smart enough to be aligned to cats in the way that we are, using LLM components, and giving it any power at all would be wildly unsafe, and also is likely impossible. The big problem is that I don’t have any proposal for how to measure whether a higher intelligence is is aligned to a lower intelligence other than just letting the higher intelligence run roughshod over a world and observing what it does, so checking that an AI is aligned to cats is not any easier than checking if an AI is aligned to humans.
I have issues with this: I don’t think you can claim that wildcats of the stone age would be pleased with what we’ve done to domestic cats either, sticking them in tiny territories where they cannot roam, kingdoms of a cage. I’m not sure using human judgement in this matter is very useful as we don’t have a good concept of what other species value.
I don’t think evolutionary pressure is an intelligence; it’s in the name. It’s a pressure, like air pressure or water pressure, not an intelligence. There is no agency. The end results can be marvelous all the same. Evolution appears to make FDT-style trades, but is actually completely myopic. It’s survival of the survivors. If you’re dead, we don’t see you.
Also sociality in spiders has evolved independently at least twenty times, and keeps going extinct. It’s probably an evolutionary dead end due to inbreeding [1]. Evolution is completely myopic.
We do, however, build rooms full of cats and catnip.
I don’t think we are materially disagreeing here, just working with very slight differences in definition of intelligence. I see a strong analogy to the debate between whether LLMs think or LLMs just predict the next token. I think that your claims that “There is no agency” and evolution is “completely myopic” are true for some reasonable definitions of agency and myopia, as long as you don’t make any arguments like “There is no agency / complete myopia, therefore evolution can’t X.” You aren’t making any such arguments, hence the lack of material disagreement.
On the subject of cats being deprived of roaming and hopped up on catnip, yeah I would not love that either, but the contrast to the outcomes for cows or chickens is big. Getting alignment close enough that the result is preferable to going extinct is a high bar. To reiterate, we could also just stop, but that would be ludicrous.
Just one factor, but the life expectancy of domestic dogs and cats is generally higher than their wild progenitors. I agree we can’t know for sure, but I would guess this with limitless food and good healthcare, and less worry about being attacked at night would mean the subjective wellbeing of domesticated cats and dogs is higher than the wild ones, despite less freedom.