Just this guy, you know?
Dagon
Karma: 12,938
What about an AI which cares a tiny bit about keeping humans alive
This is quite believable in the short (a few generations of humans) term. It’s much less so in the long term. I model “caring about a lot of things” as an equilibrium case of competing reasons for action, and almost nothing that is “care a little bit” is going to be stable.
I suppose “care a lot about maintaining a small quantity of biologicals” is possible as well for inscrutable AI reasons, but that doesn’t bring any civilization back.
Didn’t downvote only because it was already negative enough. This didn’t add anything to my understanding of your beliefs, or of any general approach to preferences or decision-making.
Extreme hypotheticals don’t tell us much. They can be useful as extrapolations or to test how scalable our intuitions are (note: they rarely are), but they don’t work as a starting point. More importantly, this argument hinges on definition of happiness, and the unstated (and incorrect, IMO) assumption that it’s time-consistent and reflectively available. In truth, people are very bad at predicting their future happiness level, and even worse at ascribing causality to that happiness.
It’s very difficult for me to come up with believable scenarios which include disempowerment and death of a majority of humans, but don’t eventually kill everyone. The uncontacted tribes (from what I understand, not truly unknown, just prefer minimal contact) may last a few hundred years in these scenarios, but biological civilization doesn’t come back.
Do you disagree with my model, or do you think I should add those remarks to the post ?
I fully agree with the distinction between pleasure and happiness, though I suspect the relationship between them is more complicated than long vs short terms. You should probably retitle the post and remove hedonic adaptation as your primary cause for the distinction. Either explore other causes, or just describe the weakness of the correlation without naming the reason.
Sure. I’ll take 2 of your 3 examples:
If peoples like taking drugs so much, why are drug addict’s less happy than an average human?
This really does not seem to be related to adaptation, but to multiple dimensions and timeframes of happiness. This is a distinction between pleasure and happiness.
Why don’t you like simple pleasures any more, if you enjoyed them when you was child (think about any computer game from your childhood)?
I definitely still like simple pleasures, they’re just not enough for happiness anymore. In fact, they never were, but memory is a funny thing. This is also not caused by adaptation but is more about different context and complexity of experience to integrate the simple pleasure into.
I sometimes argue that even the common drive to variety is distinct from hedonic adaptation, but I’ll leave that for later.
Note that I don’t deny that hedonic adaptation is real and important—people’s short-term happiness change and (partial) reversion to previous levels when losing a limb or winning a lottery are pretty strong examples. I only claim that there are multiple other mechanisms that explain a lot of change over time in experience->happiness causality.
I think this attributes way too much to hedonic adaptation. It’s a real thing, and one’s felt-happiness-from-experiences does change over time. HOWEVER, there’s also a difference between pleasure and happiness, and for each of them different nonlinear impact over different timeframes.
These more fundamental differences in conflicting types and timeframes of desirable experiences can outweigh hedonic adaption by quite a bit. Reflectively, hedonic adaptation explains less than 1⁄4 of my experenced variance in desirability of different framings of experiences.
The system is never going to be all that great—it’s really lightweight, low-information, low-committment to cast a vote. That’s a big weakness, and also a requirement to get any input at all from many readers.
It roughly maps to “want to see more of” and “want to see less of” on LessWrong, but it’s noisy enough that it shouldn’t be taken too literally.
I tend to think that all the virtues are described over-simply and that we’d do well to consider that their actual value is contextual, but their signaling value is anti-contextual (it’s more valuable for others to think you will exhibit the virtue in places where it’s probably detrimental to you to actually do so).
Thanks for the write-up, and I’d recommend you add “self-honesty” to the list, in terms of what one believes about oneself, regardless of how accurate or forthright one is to any given external audience.
I’m a huge fan, especially for the user-specific, ephemeral uses like you describe. “Summarize the major contrasting views to this post” would be awesome. I’m much less happy with publication and posting-support uses that would be the obvious things to do.
I think this Is a reasonable model for many kinds of approval/opprobrium. I think it’s a feature, rather than a bug, for a lot of topics.
Especially for those things that are mixed-sum collections of games, collusion is a valuable strategy, and out unclear which way the causality goes: is this the reason that collusion works, or is collusion just another element of strategy in the social game (s)?
ideology of “law and order,” the belief that more numerous and stricter laws lead to a stronger rule of law
Umm, do you have a cite or example of someone saying that NUMBER of laws is the key to law and order? All the discussion I’ve seen has been about enforcement of major categories of misbehavior, and addition of fairly broad, unspecific norms that are not currently in law.
In many places, the fine-grained detailed regulation and legislation is a different pathology, more pseudo-technocratic, not ‘law and order’.
A person in the future might perceive the privilege of receiving life saving chemotherapy almost surely as a totally nasty and brutish treatment,
Plenty of people in the present recognize this, and choose not to endure it. I would argue that this choice and the respect of differing opinions on related topics is fairly modern, and I strongly prefer the current equilibrium. I recognize that circumstances will change, and the tension between enforcing conformity vs accepting inefficiency will shift as a result. My preference isn’t a universal value judgement.
I think that’s the key area where I’m not sure if we agree, or if we just are somewhat aligned accidentally: I don’t think there is any objective valuation of individual or group behaviors. Many of us have preferences, but that’s an overlay of individual valuation on top of each of our framings of our experiences. This does include some pretty strong modeling of causality, especially in smaller subsets of space-time. But it doesn’t actually map to any territory.
I’m not sure if you’re arguing against a simplistic “progress toward utopia” narrative (which seems obviously wrong), or against all narrative descriptions of the complex weave of individuals across the past (which I’m not sure what other options there are).
I have done some amount of thinking on it, and I consider myself very lucky to have been born in the circumstances that I find myself in. It’s not uniform, but generally “nasty, brutish, and short” does describe most lives before the 20th century. And honestly, most today—I’m lucky on more dimensions than just the historical period I’m in.
This is far different than progress being inevitable or monotonic. In a lot of ways the end of the last century was better than the current (for my personal lived experiences). But it is progress, and it’s pretty significant when looked at in half-century chunks for the last half-millennium. It’s much less clear about fine-grained improvements over longer timeframes.
I suspect it matters quite a bit WHICH 10-20 people support the project. And I further suspect that the right people are unlikely to sell their time in this way.
That’s not to say that it’s a bad idea to offer to compensate people for the time and risk of analyzing your proposal. But you should probably filter for pre-interest with a brief synopsis of what you actually want to do (not “get funding”, but “perform this project that does …”).
The United States has been facing these tough decisions for quite some time, and so far has chosen not to make them. There’s a great deal of ruin in a nation.
Thanks for the clarification—I’m still a bit unsure if “planetary civilization” is distinct from “the specific set of individuals inhabiting a planet”, and I should admit that I’m highly skeptical of the value (to an AGI or even to other humans) of a specific individual’s brain-state, and I have a lot of trouble following arguments that imply migration or resurrection of more than a few percent of biological intelligences.
This doesn’t seem helpful or likely to get useful engagement on LessWrong. Some specifics about where you’re encountering these ideas and why the obvious strategy of “ignore them and go do something useful” isn’t applicable would go a long way.
Can you explain your model for what “survive” and “resurrect” means for a civilization, as opposed to individuals that happen to exist within a civilizational context? Relatedly, what’s your model for a civilization’s decision theory that makes “random strategy” a coherent idea?
My model is that a civilization is an emergent set of behaviors and expectations of individuals that are coexistent in time and space. And I’m not sure your thinking is applicable on that level.
Infinite time to run does NOT imply that every possible local configuration will obtain. If it’s cyclical, it could repeat the same few trillion years, which is a TINY subset of possible-experience-entity-space.
”Most things happen at the wrong time, and many things don’t happen at all. The conscientious historian will correct these defects.”—Heroditus
I think my response is above—I have no intention to argue either side of that. It’s so far out of what’s possible that there’s really no information available about how one should or could react.
In truth, I would advise you to say no (or just walk away and don’t engage) if someone made you this offer—they’re dangerously delusional or trying to scam you.