Put it this way, if there was a pill that I believed would cause me to effectively have that goal, in a way that was compatible with a livable life, I would take it.
But don’t you already have this pill? You know, you can just do what you want. There is no akrasia fairy that forces you to procrastinate. Isn’t that basic reductionism? You are an algorithm, that algorithm optimizes for a certain state, we call this state its goal. An algorithm just is its code, so it can only optimize for this goal. It is incoherent to say that the algorithm does A, but wants B. The agent is its behavior.
So, how could you not do what you want? Your self-modelling can be deficient or biased, but part of the claim is that this bias actually helps you signal better, and is thus advantageous. Or you might not be very powerful and choose sub-optimal options, but that’s also not the claim. How, algorithmically, does your position work?
(The best I can do is to assume that there are two agents A and B, how want X and Y, respectively, and A is really good at getting X, but B unfortunately models itself as being A, but is also incompetent enough to think A wants Y, so that B still believes it wants Y. B has little power and is exploited by A, so B rarely makes progress towards Y and thus has a problem and complains. But that doesn’t sound too realistic.)
There are many modules, running different algorithms. I identify with my conscious modules, which quite often lose out to the non-conscious ones.
I find the claim “you are the sum of your conscious and non-conscious modules, so whatever they produce as their overall output is what you want” to be rather similar to the claim that “you are the sum of your brain and body, so whatever they produce as their overall output is what you want”. Both might be considered technically true, but it still seems odd to say that a paraplegic could walk if he wanted to, and him not walking just demonstrates that he doesn’t really want to.
While we’re at it, there’s also the claim that I am the sum of the conscious and unconscious modules of everyone living in Massachusetts. And an infinite number of other claims along those lines.
Many of these sorts of claims seem odd to me as well.
Both might be considered technically true, but it still seems odd to say that a paraplegic could walk if he wanted to, and him not walking just demonstrates that he doesn’t really want to.
There is a difference between preferences and constraints. See e.g. Caplan.
Hmm. It occurs to me that this disagreement might be because my original description of the issue mentioned several different scenarios, which I did not clearly differentiate between.
Scenario one: a person who wants to do prestigious jobs for a charity. Depending on the person, it could be that this is genuinely his preference, which won’t change even if consciously realizes that this is his preference. In that scenario, then yes, it’s just a preference and there’s no problem as such. (Heck, I know I could never get any major project done if there wasn’t some status pull involved in the project, somehow.) On the other hand, the person might want to change his behavior if he realized how and why he was acting.
Scenario two: a person wants to e.g. graduate from school, but he’s having a hard time studying effectively because he isn’t fully motivated on a subconscious level. This would correspond to what Caplan defines as a constraint:
If a person had 24 hours of time to divide between walking and resting, and a healthy person faced budget constraint A, then after contracting the flu or cancer, the same person would face a budget constraint such as B. A sufficiently sick person might collapse if he tried to walk for more than a few miles – suffering from reduced endurance as well as reduced speed. Then the budget constraint of the sick person would differ more starkly from the healthy person’s, as shown by the kinked constraint in Figure 2.
This person tries to get studying done, and devotes a lot of time and energy to it. But because his subconscious gaols aren’t fully aligned with his conscious goals, he needs to allocate far more time and energy to studying than the person whose subconscious goals are fully aligned with his conscious goals.
I don’t think the second kind is really a constraint. It’s more like the ADD child example Caplan uses:
A few of the symptoms of inattention [...] are worded to sound more like constraints. However, each of these is still probably best interpreted as descriptions of preferences. As the DSM uses the term, a person who “has difficulty” “sustaining attention in tasks or play activities” could just as easily be described as “disliking” sustaining attention. Similarly, while “is often forgetful in daily activities” could be interpreted literally as impaired memory, in context it refers primarily to conveniently forgetting to do things you would rather avoid. No one accuses a boy diagnosed with ADHD of forgetting to play videogames.
I can easily frame the student as disliking studying (for good reasons—it’s hard work and probably pretty useless for their goals) and thus playing up the pain. This episode of struggle and suffering itself is useful, so they keep it up. Why should I conclude that this is a problematic conflict and not a good compromise? And even if I accept the goal conflict, why side with the lamenting part? Why not with the part that is bored and hates the hard work, and certainly doesn’t want to get more effective and study even more?
(I think I have clarified my position enough and the attempts by others haven’t helped me to understand your claim. I don’t want to get into an “I value this part” debate. These have never been constructive before, so I’m going to drop it now.)
But don’t you already have this pill? You know, you can just do what you want. There is no akrasia fairy that forces you to procrastinate. Isn’t that basic reductionism?
The best I can do is to assume that there are two agents A and B, how want X and Y, respectively, and A is really good at getting X, but B unfortunately models itself as being A, but is also incompetent enough to think A wants Y, so that B still believes it wants Y. B has little power and is exploited by A, so B rarely makes progress towards Y and thus has a problem and complains. But that doesn’t sound too realistic
That actually sounds like a pretty good description of the problem, and of “normal” human behavior in situations where X and Y aren’t aligned. (Which, by the way, is not a human universal, and there are good reasons to assume that it’s not the only kind of situation for which evolution has prepared us).
The part that’s missing from your description is that part A, while very persistent, lacks any ability to really think things through in the way that B can, and makes its projections and choices based on a very “dumb” sort of database.… a database that B has read/write access to.
The premise of mindhacking, at least in the forms I teach, is that you can change A’s behavior and goals by tampering with its database, provided that you can find the relevant entries in that database. The actual tampering part is pretty ridiculously easy, as memories are notoriously malleable and distortable just by asking questions about them. Finding the right memories to mess with is the hard part, since A’s actual decision-making process is somewhat opaque to B, and most of A’s goal hierarchy is completely invisible to B, and must be inferred by probing the database with hypothetical-situation queries.
One of the ways that A exploits B is that B perceives itself as having various overt, concrete goals… that are actually comparatively low-level subgoals of A’s true goals. And as I said, those goals are not available to direct introspection; you have to use hypothetical-situation queries to smoke out what A’s true goals are.
Actually, it’s somewhat of a misnomer to say that A exploits B, or even to see A as an entity at all. To me, A is just machinery, automated equipment. While it has a certain amount of goal consistency protection (i.e., desire to maintain goals across self-modification), it is not very recursive and is easily defeated once you identify the Nth-order constraint on a particular goal, for what’s usually a very low value of N.
So, it’s more useful (I find) to think of A as a really powerful and convenient automaton that can learn and manage plenty of things on its own, but which sometimes gets things wrong and needs B’s help to troubleshoot the problems.
That’s because part A isn’t smart enough to resolve inter-temporal conflicts on its own; absent injunctive relief or other cached thoughts to overcome discounting, it’ll stay stuck in a loop of preference reversals pretty much forever.
Yes. As far as I can tell, you already have the option, but don’t use it. What makes you think you would do so in future cases? If akratics reliably would take such a pill, wouldn’t you expect self-help to work? The phenomenon of people getting results, but still not sticking with it shouldn’t exist then.
If akratics reliably would take such a pill, wouldn’t you expect self-help to work?
My own observation is that people generally stop using self-help techniques that actually work, and often report puzzlement as to why they stopped.
So I think akratics would take such a pill. The catch is that self-help is generally a pill that must be taken daily, and as soon as your brain catches up with the connection between taking the pill and making progress on a goal you don’t actually want to make progress on… you’ll start “mysteriously forgetting” to take the pill.
The only thing I know that works for this sort of situation is getting sufficiently clear on your covert goals to resolve the conflict(s) between them.
It’s excessive to claim that the hard work, introspection, and personal -change- (the hardest part) required to align your actions with a given goal are equivalent in difficulty or utility to just taking a pill.
Even if self-help techniques consistently worked, you’d still have to compare the opportunity cost of investing that effort with the apparent gains from reaching a goal. And estimating the utility of a goal is really difficult, especially when it’s a goal you’ve never experienced before.
Put it this way, if there was a pill that I believed would cause me to effectively have that goal, in a way that was compatible with a livable life, I would take it.
But don’t you already have this pill? You know, you can just do what you want. There is no akrasia fairy that forces you to procrastinate. Isn’t that basic reductionism? You are an algorithm, that algorithm optimizes for a certain state, we call this state its goal. An algorithm just is its code, so it can only optimize for this goal. It is incoherent to say that the algorithm does A, but wants B. The agent is its behavior.
So, how could you not do what you want? Your self-modelling can be deficient or biased, but part of the claim is that this bias actually helps you signal better, and is thus advantageous. Or you might not be very powerful and choose sub-optimal options, but that’s also not the claim. How, algorithmically, does your position work?
(The best I can do is to assume that there are two agents A and B, how want X and Y, respectively, and A is really good at getting X, but B unfortunately models itself as being A, but is also incompetent enough to think A wants Y, so that B still believes it wants Y. B has little power and is exploited by A, so B rarely makes progress towards Y and thus has a problem and complains. But that doesn’t sound too realistic.)
There are many modules, running different algorithms. I identify with my conscious modules, which quite often lose out to the non-conscious ones.
I find the claim “you are the sum of your conscious and non-conscious modules, so whatever they produce as their overall output is what you want” to be rather similar to the claim that “you are the sum of your brain and body, so whatever they produce as their overall output is what you want”. Both might be considered technically true, but it still seems odd to say that a paraplegic could walk if he wanted to, and him not walking just demonstrates that he doesn’t really want to.
While we’re at it, there’s also the claim that I am the sum of the conscious and unconscious modules of everyone living in Massachusetts. And an infinite number of other claims along those lines.
Many of these sorts of claims seem odd to me as well.
There is a difference between preferences and constraints. See e.g. Caplan.
Hmm. It occurs to me that this disagreement might be because my original description of the issue mentioned several different scenarios, which I did not clearly differentiate between.
Scenario one: a person who wants to do prestigious jobs for a charity. Depending on the person, it could be that this is genuinely his preference, which won’t change even if consciously realizes that this is his preference. In that scenario, then yes, it’s just a preference and there’s no problem as such. (Heck, I know I could never get any major project done if there wasn’t some status pull involved in the project, somehow.) On the other hand, the person might want to change his behavior if he realized how and why he was acting.
Scenario two: a person wants to e.g. graduate from school, but he’s having a hard time studying effectively because he isn’t fully motivated on a subconscious level. This would correspond to what Caplan defines as a constraint:
This person tries to get studying done, and devotes a lot of time and energy to it. But because his subconscious gaols aren’t fully aligned with his conscious goals, he needs to allocate far more time and energy to studying than the person whose subconscious goals are fully aligned with his conscious goals.
I don’t think the second kind is really a constraint. It’s more like the ADD child example Caplan uses:
I can easily frame the student as disliking studying (for good reasons—it’s hard work and probably pretty useless for their goals) and thus playing up the pain. This episode of struggle and suffering itself is useful, so they keep it up. Why should I conclude that this is a problematic conflict and not a good compromise? And even if I accept the goal conflict, why side with the lamenting part? Why not with the part that is bored and hates the hard work, and certainly doesn’t want to get more effective and study even more?
(I think I have clarified my position enough and the attempts by others haven’t helped me to understand your claim. I don’t want to get into an “I value this part” debate. These have never been constructive before, so I’m going to drop it now.)
No.
That actually sounds like a pretty good description of the problem, and of “normal” human behavior in situations where X and Y aren’t aligned. (Which, by the way, is not a human universal, and there are good reasons to assume that it’s not the only kind of situation for which evolution has prepared us).
The part that’s missing from your description is that part A, while very persistent, lacks any ability to really think things through in the way that B can, and makes its projections and choices based on a very “dumb” sort of database.… a database that B has read/write access to.
The premise of mindhacking, at least in the forms I teach, is that you can change A’s behavior and goals by tampering with its database, provided that you can find the relevant entries in that database. The actual tampering part is pretty ridiculously easy, as memories are notoriously malleable and distortable just by asking questions about them. Finding the right memories to mess with is the hard part, since A’s actual decision-making process is somewhat opaque to B, and most of A’s goal hierarchy is completely invisible to B, and must be inferred by probing the database with hypothetical-situation queries.
One of the ways that A exploits B is that B perceives itself as having various overt, concrete goals… that are actually comparatively low-level subgoals of A’s true goals. And as I said, those goals are not available to direct introspection; you have to use hypothetical-situation queries to smoke out what A’s true goals are.
Actually, it’s somewhat of a misnomer to say that A exploits B, or even to see A as an entity at all. To me, A is just machinery, automated equipment. While it has a certain amount of goal consistency protection (i.e., desire to maintain goals across self-modification), it is not very recursive and is easily defeated once you identify the Nth-order constraint on a particular goal, for what’s usually a very low value of N.
So, it’s more useful (I find) to think of A as a really powerful and convenient automaton that can learn and manage plenty of things on its own, but which sometimes gets things wrong and needs B’s help to troubleshoot the problems.
That’s because part A isn’t smart enough to resolve inter-temporal conflicts on its own; absent injunctive relief or other cached thoughts to overcome discounting, it’ll stay stuck in a loop of preference reversals pretty much forever.
Are you saying that I would not take such a pill if it were offered to me in pill form, and my prediction that I would is wrong, or something else?
Yes. As far as I can tell, you already have the option, but don’t use it. What makes you think you would do so in future cases? If akratics reliably would take such a pill, wouldn’t you expect self-help to work? The phenomenon of people getting results, but still not sticking with it shouldn’t exist then.
My own observation is that people generally stop using self-help techniques that actually work, and often report puzzlement as to why they stopped.
So I think akratics would take such a pill. The catch is that self-help is generally a pill that must be taken daily, and as soon as your brain catches up with the connection between taking the pill and making progress on a goal you don’t actually want to make progress on… you’ll start “mysteriously forgetting” to take the pill.
The only thing I know that works for this sort of situation is getting sufficiently clear on your covert goals to resolve the conflict(s) between them.
I was definitely envisaging a pill that only needs to be taken once, not one that needs to be taken daily.
It’s excessive to claim that the hard work, introspection, and personal -change- (the hardest part) required to align your actions with a given goal are equivalent in difficulty or utility to just taking a pill.
Even if self-help techniques consistently worked, you’d still have to compare the opportunity cost of investing that effort with the apparent gains from reaching a goal. And estimating the utility of a goal is really difficult, especially when it’s a goal you’ve never experienced before.
The backstabbing AI would take the non-backstabbing pill.