Stuart_Armstrong
Karma: 20,430 (LW), 183 (AF)
NewTop
Humans interpreting humans
Anchoring vs Taste: a model
I don’t know which problems/systems you’re referring to. Maybe you could cite these in the post to give more motivation?
The main one is when I realised the problems with CEV: https://www.lesswrong.com/posts/vgFvnr7FefZ3s3tHp/mahatma-armstrong-ceved-to-death
The others are mainly oral, with people coming up with plans that involve simulating humans for long periods of time, me doing the equivalent of saying “have you considered value drift” and (often) the reaction from the other revealing that no, they had not considered value drift.
Because the difference is large between what the setup will be in practice, and what current research is in practice.
What are the most important differences that you foresee?
The most important differences I foresee are the unforseen :-) I mean that seriously, because anything that is easy to foresee will possibly be patched before implementation.
But if we look at how research happens nowadays, it has a variety of different approaches and institutional cultures, certain levels of feedback both from within the AI safety community and the surrounding world, grounding our morality and keeping us connected to the flow of culture (such as it is).
Most of the simulation ideas do away with that. If someone suggested that the best idea for AI safety would be to lock up AI safety researchers in an isolated internet-free house for ten years and see what they came up with, we’d be all over the flaws in this plan (and not just the opportunity costs). But replace that physical, grounded idea with a similar one that involves “simulation”, and suddenly people flip into far mode and are more willing to accept it. In practice, a simulation is likely to be far more alien and alienating that just locking people up in a house. We have certain levels of control in a simulation that we wouldn’t have in reality, but even that could hurt—I’m not sure how I would react if I knew my mind and emotions and state of tiredness were open to manipulation.
So what I’m mainly trying to say is that using simulations (or predictions about simulations) to do safety work is a difficult and subtle project, and needs to be thoroughly planned out with, at minimum, a lot of psychologists and some anthropologists. I think it can be done, but not glibly and not easily.
Also, on a more minor note, I expect that if I try and preserve myself from value drift, using only the resources I had in the simulation—I expect to fail. Social dynamics might work though, so we do need to think about those.
That they make some sensible points, but they’re wrong when they push them to far (and that they are mixing factual truths with preferences a lot). Christians do have their own “truths”, if we interpret these truths as values, which is what they generally are. “It is a sin to engage in sex before marriage” vs “(some) sex can lead to pregnancy”. If we call both of these “truths”, then we have a confusion.
Because I’ve already found problems with these systems in the past few years, problems that other people did not expect there to be. If one of them had been put into such a setup then, I expect that it would have failed. Sure, if current me was put in the system, maybe I could find a few more problems and patch them, because I expect to find them.
But I wouldn’t trust many others, and I barely trust myself. Because the difference is large between what the setup will be in practice, and what current research is in practice. The more we can solve these issues ahead of time, the more we can delegate.
Would I think for ten thousand years?
“Normative assumptions” need not be complex
I think that a model of an individual’s preferences is likely to be better represented by taking multiple approaches, where each fails differently.
I agree. But what counts as a failure? Unless we have a theory of what we’re trying to define, we can’t define failure beyond our own vague intuitions. But once we have a better theory, defining failure becomes a lot easier.
Cheers!
Even without self-flagellation, if your marginal utility per $ is much lower, and you don’t use your own surplus in a fungible way to donate/buy more, donating can be much higher impact than trade. First of all, you have more freedom to target donations that trade, and even if we ignore that, capturing all your money is better for the producer than just capturing the producer surplus (and the marginal utility of the consumer surplus to you is sufficiently low that adding it on doesn’t bring the surpluses to a higher number).
I haven’t seen anyone on Less Wrong argue against CEV as a vision for how the future of humanity should be determined.
Well, now you’ve seen one ^_^ : https://www.lesswrong.com/posts/vgFvnr7FefZ3s3tHp/mahatma-armstrong-ceved-to-death
I’ve been going on about the problems with CEV (specifically with extrapolation) for years. This post could also be considered a CEV critique: https://www.lesswrong.com/posts/WeAt5TeS8aYc4Cpms/values-determined-by-stopping-properties
Shame :-(
My impression is that in full generality it is unsolvable, but something like starting with an imprecise model of approval / utility function learned via ambitious value learning and restricting explanations/questions/manipulation by that may be work.
Yep. As so often, I think these things are not fully value agnostic, but don’t need full human values to be defined.
possibility that explanation can be usefully defined in a way that distinguishes it from manipulation?
I think explanation can be defined (see https://agentfoundations.org/item?id=1249 ). I’m not confident “explanation with no manipulation” can be defined.
Mainly agree, but I’ll point out that addicts at different moment can prefer to not have heroin—in fact, as a addict of much more minor things (eg News), I can testify that I’ve done things I knew I didn’t want to do at every moment of the process (before, during, and after).
Given some definition of corrigibility, yes.
Humans have beliefs and values twisted together in all kinds of odd ways. In practice, increasing our understanding tends to go along with having a more individualist outlook, a greater power to impact the natural world, less concern about difficult-to-measure issues, and less respect for traditional practices and group identities (and often the creation of new group identities, and sometimes new traditions).
Now, I find those changes to be (generally) positive, and I’d like them to be more common. But these are value changes, and I understand why people with different values could object to them.
Thanks for writing that post; have you got much in terms of volunteers currently?
Just want to thank you for doing these newsletters...