I agree Dagon—and I actually specifically discussed this issue in the article I referenced in the comment I posted just before this one. Part of what I said was: “There may be one way that we could deal with this issue, and that would be to use different language to describe choices. Conventionally, if I have just picked up a glass we would say that I chose to pick it up. This whole idea of ‘choosing’ can cause us cognitive difficulties. Maybe it would be better to consider my ‘choice’ to pick up the glass as really ‘finding out’ that I was predisposed to pick it up.” I also agree with FAWS said—that this is implied by “decision” anyway—at least to anyone who thinks about it enough.
PaulAlmond
SilasBarta, yes. I decided to change to this username as it is more obvious who I am. I generally use my real name in online discussions of this type: I have it on my website anyway. I don’t envisage using the PaulUK name again.
(This is my new username. I was formerly PaulUK.) Just a quick note to say that, after leaving the “Minds, Measure, Substrate and Value” series for a while, I am currently doing Part 4, which will deal with some of the objections that have been made by, among others, Less Wrong members, Part 5, hopefully, will generalize into a cosmological view, as opposed to one that is just about minds.
I don’t see this as being much of an issue for getting usable AI working: it may be an issue if we demand perfect modeling of reality from a system, but there is no reason to suppose we have that.
As I see it, we can set up a probabilistic model of reality and extend this model in an exploratory way. We would continually measure the relevance of features of the model—how much effect they have on predicted values that are of interest—and we would tend to keep those parts of the model that have high relevance. If we “grow” the model out from the existing model that is known to have high relevance, we should expect it to be more likely that we will encounter further, high-relevance “regions”.
“Due to an unexpected mental glitch, he threatens Joe again. Joe follows his disposition and ignores the threat. BOOM. Here Joe’s final decision seems as disastrously foolish as Tom’s slip up.”
But of course, the initial decision to take the pill may be rational, and the “final decision” is constrained so much that we might regard it as a “decision” in name only. The way I see it: When Joe takes the pill, he will stop rational versions of Tom from threatening him, meaning he benefits, but will be at increased risk of irrational versions of Tom threatening him, meaning he loses. Whether the decision to take the pill is rational depends on how many rational versions of Tom he thinks are out there and how many irrational ones there are, as well as the relative costs of being forced to shine shoes and being blown up. If Toms tend to be rational, and shining shoes is unpleasant enough, taking the pill may be rational.
This kind of scenario has made me think in the past: Could this have contributed to some of our emotional tendencies? At times, we experience emotions that over-ride our rational behavior. Anger is a good example, though gratitude might be as well. There may be times when it is not just rational, in terms of reward and cost, to hit back at someone who has wrong us, but we may do anyway because we are angry. However, if we never got angry, and acted rationally all the time, we may be easy targets for people who know that they can wrong us and then retreat to some safe situation where revenge would be irrational. Something that can reduce our rationality, so that we act even when it is not in our interests, might, almost paradoxically, be a good thing for us, because it would make it less rational to attack us like this in the first place. Maybe anger is partly there for that reason—literally to ensure that we will actually do things that get ourselves killed to hit back at someone, as a deterrent.
Of course, someone could ask how people are supposed to know we have that tendency—but when people saw anger working in themselves and others they would generally get the idea—they would understand the consequences of reduced rationality in some situations. It could be argued that the best strategy is to fake your ability to become angry. Maybe you become angry in trivial situations, where the cost of the anger is minimal, while in the extreme situation where you are likely to get killed you act rationally, but a problem with this is that it is more complicated behavior, so we might assume that it is harder for it get evolved in the first place. There would presumably be some kind of balance between real deterrence and fake deterrence at work here.
I can think of real-world examples of this “pill”. I think there is supposed to be one wealthy person who told his family that if he was kidnapped a ransom was not to be paid under any circumstances. Now, clearly, his family are likely to ignore that and pay: Any deterrence has failed and the rational thing is to save his life. That suggests that he may have taken precautions: He may have done his best to make it impossible for his family to pay a ransom.
Is anyone going to propose this as an answer to (what some say is) the Fermi paradox?
We are looking at very long time scales here, so how wide should our scope be? If we use a very wide scope like this, we get issues, but if we widen it still further we might get even more. Suppose the extent of reality were unlimited, and that the scope of effect of an individual action were unlimited, so that if you do something it affects something, which affects something else, which affects something else, and so on, without limit. This doesn’t necessarily need infinite time: We might imagine various cosmologies where the scope could be widened in other ways. Where would that leave the ethical value of any action we commit?
I will give an analogy, which we can call “Almond’s Puppies” (That’s a terrible name really, but it is too late now.)
Suppose we are standing at the end of two lines of boxes. Each line continues without end, and each box contains a puppy—so each line contains an infinity of puppies. You can choose to press a button to blow up the first box or another button to spare it. After you press the button, some mechanism, that you can’t predict, will decide to blow up the second box or spare it, based on your decision, and then it will decide to blow up the third box or spare it, based on your decision, and so on. So you press that button, and either the first box is blown up or spared, and then boxes get blown up or spared right along the line, with no end to it.
You have to press a button to start one line off. You choose to press the button to spare the first puppy. Someone else chooses to press the button to blow up the first puppy. The issue now is: Did the other person do a bad thing? If so, why? Did he kill more puppies than you? Does the fact that he was nicer to the nearby puppies matter? Does it matter that the progress of the wave of puppy explosions along the line of boxes will take time, and at any instant of time, only a finite number of puppies will have been blown up, even though there is no end to it in the future?
If we are looking at distant future scenarios, we might ask if we are sure that reality is limited.
“There’s a relationship to your decision but you don’t know which one”. You won’t see all the puppies being spared or all the puppies being blown up. You will see some of the puppies being spared and some of them being blown up, with no obvious pattern—however you know that your decision ultimately caused whatever sequence of sparing/blowing up the machine produced.
I think Occam’s razor is fairly easy to justify in general.
I would say that the problem is resolved if we assume that our reference class is “every conceivable, formally describable universe”—and the description in “formally describable” doesn’t just describe the universe’s state at an instant: It describes the entire universe, and its history, as an object. We should assume that one of those objects corresponds to our world.
Once we have that, we have some data from experiments, and we generate a partial model, which is an algorithm that accepts past data and predicts future data. The model needs to work for past data, and we want one that is as likely as possible to predict future data correctly. We are hoping that our partial model correctly describes the behavior of our universe—which is one of the ones in the reference class. The greater the information content in the partial model, the more specific it is being, and the smaller the proportion of possible universes in the reference class that will agree with it. The smaller the information content in the partial model, the more general it is being, and the greater the proportion of possible universes in the reference class that will agree it and, therefore, the greater will be the chance that one of these universes happens to be the real one (or the one in which we are living if you subscribe to some form of modal realism—not that it matters whether you do or not).
This should easily deal with the issue of why, when you see ordered behavior in reality, you should expect to see continued ordered behavior. It doesn’t resolve these issues:
Why do we see any ordered behavior in the first place? None of this imposes any ordered behavior on reality. It simply says that if you see some you should expect more to follow. Any simplicity that you observe does not imply that reality is simple: it simply means that your partial model is relying on a simple feature of reality that happened to be there—a very different thing. It does nothing to stop reality being a chaotic mess. However, an anthropic argument might be used here.
It doesn’t resolve the issue of the coding system used for the descriptions—although I think that issue can be resolved without too much trouble—though I am sure others would disagree.
I disagree with the idea that modal realism, whether right or not, changes the chances of any particular hypothesis like that being true. I am not saying that we can never have a rational belief about whether or not modal realism is true: There may or may not be a philosophical justification for modal realism. However, I do think that whether modal realism applies has no bearing on the probability of you being in some situation, such as in a computer simulation. I think this issue needs debating, so for that purpose I have asserted this is a rule, which I call “The Principle of Modal Realism Equivalence”, and that gives us something well-defined to argue for or against. I define and assert the rule, and give a (short) justification of it here: http://www.paul-almond.com/ModalRealismEquivalence.pdf.
A kind of funny way in which something like this might (just about) happen in reality occurs to me: Possible time delay in human awareness of decision making. Suppose when you make a conscious decision, your brain starts to become committed to that decision before you become aware of it, so if you suddenly decide to press a button then your brain was going through the process of committing you to pressing it before you actually knew you were going to press it. That would mean that every time you took a conscious decision to act, based on some rational grounds, you should really have been wanting to be the person who had been predisposed to act in that way a short time ago, when the neural machinery was pushing you towards that decision. I’m not saying this resolves any big issues, but maybe it can be amusingly uncomfortable for a few people—especially given some (admittedly controversial) experiments. In fact, with some brainwave monitoring equipment, a clever experiment design, and a very short experiment duration, you might even be able to set up something slightly resembling Newcomb’s paradox!
I have a description here of a practical demonstration of Newcomb’s paradox that might just be possible, with current or near-future technology. It would rely, simply, on the brain being more predictable over a short span of time. I would be interested to see what people think about the feasibility.
A test subject sits at a desk. On the desk are two buttons. On button “O” corresponds to opening one box. The other button “B” corresponds to opening both boxes. There is a computer, with a display screen. The boxes are going to be computer simulated: A program in the computer has a variable for the amount of money in the each box.
This is how an experimental run proceeds.
The subject sits at the desk for some random amount of time, during which nothing happens.
A “Decision Imminent” message appears on the computer screen. This is to warn the subject that his/her decision is about to be demanded imminently.
A short time after (maybe a second or two, or a few seconds), the computer program decides how much money will go in each box, and it sets the variables accordingly, without showing the user. As soon as that is done, a “Select a box NOW” message appears on the computer screen. The subject now has a (very) limited amount of time to press either button “O” or “B” to select one or both boxes. The subject will have to press one of the buttons almost immediately before the offer is withdrawn.
The subject is then shown the amount of money that was in each box.
Now, here is the catch (and everyone here will have guessed it).
The subject is also wired up to brain monitoring equipment, which is connected to the computer. When the “Decision imminent” message appeared, the computer started to examine the subject’s brainwaves, to try to see the decision to press being formed. Just before the “Select a box NOW” message appeared, it used this information to load the simulated boxes according to the rules of the Newcomb’s paradox game being discussed here.
I have no idea what level of accuracy could be achieved now, but it may be that some people could be made to have a worrying experience.
I wrote this some years back. It is probably not without its flaws (and got some criticism on Less Wrong), but the general idea may be relevant: http://www.paul-almond.com/WhatIsALowLevelLanguage.htm
I think this can be dealt with in terms of measure. In a series of articles, “Minds, Measure, Substrate and Value” I have been arguing that copies cannot be considered equally, without regard to substrate: We need to take account of measure for a mind, and the way in which the mind is implemented will affect its measure. (Incidentally, some of you argued against the series: After a long delay [years!], I will be releasing Part 4, in a while, which will deal with a lot of these objections.)
Without trying to present the full argument here, the minimum size of the algorithm that can “find” a mind by examining some physical system will determine the measure of that mind—because it will give an indication of how many other algorithms will exist that can find a mind. I think an AI would come to this view to: It would have to use some concept of measure to get coherent results: Otherwise it would be finding high measure, compressed human minds woven into Microsoft Windows (they would just need a LOT of compressing...). Compressing your mind will increase the size of the algorithm needed to find it and will reduce your measure, just as running your mind on various kinds of physical substrate would do this. Ultimately, it comes down to this:
“Compressing your mind will have an existential cost, such existential cost depending on the degree of compression.”
(Now, I just know that is going to get argued with, and the justification for it would be long. Seriously, I didn’t just make it up off the top of my head.)
When Dr Evil carries out his plan, each of the trillion minds can only be found by a decompression program and there must be at least a sufficient number of bits to distinguish one copy from another. Even ignoring the “overhead” for the mechanics of the decompression algorithm itself, the bits needed to distinguish one copy from another will have an existential cost for each copy—reducing its measure. An AI doing CEV which has a consistent approach will take this into account and regard each copy as not having as great a vote.
Another scenario, which might give some focus to all this:
What if Dr Evil decides to make one trillion identical copies and run them separately? People would disagree on whether the copies would count: I say they would and think that can be justified. However, he can now compress them and just have the one copy which “implies” the trillion. Again, issues of measure would mean that Dr Evil’s plan would have problems. You could add random bits to the finding algorithm to “find” each mind, but then you are just decreasing the measure: After all, you can do that with anyone’s brain.
That’s compression out of the way.
Another issue is that these copies will only be almost similar, and hence capable of being compressed, as long as they aren’t run for any appreciable length of time (unless you have some kind of constraint mechanism to keep them almost similar—which might be imagined—but then the AI might take that into account and not regard them as “properly formed” humans). As soon as you start running them, they will start to diverge, and compression will start to become less viable. Is the AI supposed to ignore this and look at “potential future existence for each copy”? I know someone could say that we just run them very slowly, so that while you and I have years of experience, each copy has one second of experience, so that during this time the storage requirements increase a bit, but not much. Does that second of experience get the same value in CEV? I don’t pretend to answer these last questions, but the issues are there.
I think I know what you are asking here, but I want to be sure. Could you elaborate, maybe with an example?
Agreed—MWI (many-worlds interpretation) does not have any “collapse”: Instead parts of the wavefunction merely become decoherent with each other which might have the appearance of a collapse locally to observers. I know this is controversial, but I think the evidence is overwhelmingly in favor of MWI because it is much more parsimonious than competing models in the sense that really matters—and the only sense in which the parsimony of a model could really be coherently described. (It is kind of funny that both sides of the MWI or !MWI debate tend to refer to parsimony.)
I find it somewhat strange that people who have problems with “all those huge numbers of worlds in MWI” don’t have much of a problem with “all those huge numbers of stars and galaxies” in our conventional view of the cosmos—and it doesn’t cause them to reach for a theory which has a more complicated basic description but gets rid of all that huge amount of stuff. When did any of us last meet anyone who claimed that “the backs of objects don’t exist, except those being observed directly or indirectly by humans because it is more parsimonious not to have them there, even if you need a contrived theory to do away with them”? That’s the problem with arguing against MWI: To reduce the “amount of stuff in reality”—which never normally bothers us with theories, and shouldn’t now, you have to introduce contrivance where it is really a bad idea—into the basic theory itself—by introducing some mechanism for “collapse”.
Somehow, with all this, there is some kind of cognitive illusion going on. As I don’t experience it, I can’t identify with it and have no idea what it is.
These worlds aren’t being “created out of nowhere” as people imagine it. They are only called worlds because they are regions of the wavefunction which don’t interact with other regions. It is the same wavefunction, and it is just being “sliced more thinly”. To an observer, able to look at this from outside, there would just be the wavefunction, with parts that have decohered from each other, and that is it. To put it another way, when a world “splits” into two worlds, it makes sense to think of it as meaning that the “stuff” (actually the wavefunction) making up that world is divided up and used to make two new, slightly different worlds. There is no new “stuff” being created. Both worlds actually co-exist in the same space even: It is only their decoherence from each other that prevents interaction. You said that your problem is “how they (the worlds) are created” but there isn’t anything really anything new being created. Rather, parts of reality are ceasing interaction with each other and there is no mystery about why this should be the case: Decoherence causes it.
This seems like pretty much Professor John Searle’s argument, to me. Your argument about the algorithm being subject to interpretation and observer dependent has been made by Searle who refers to it as “universal realizability”.
See;
Searle, J. R., 1997. The Mystery of Consciousness. London: Granta Books. Chapter 1, pp.14-17. (Originally Published: 1997. New York: The New York Review of Books. Also published by Granta Books in 1997.)
Searle, J. R., 2002. The Rediscovery of the Mind. Cambridge, Massachusetts: The MIT Press. 9th Edition. Chapter 9, pp.207-212. (Originally Published: 1992. Cambridge, Massachusetts: The MIT Press.)
I started a series of articles, which got some criticism on LW in the past, dealing with this issue (among others) and this kind of ontology. In short, if an ontology like this applies, it does not mean that all computations are equal: There would be issues of measure associated with the number (I’m simplifying here) of interpretations that can find any particular computation. I expect to be posting Part 4 of this series, which has been delayed for a long time and which will answer many objections, in a while, but the previous articles are as follows:
Minds, Substrate, Measure and Value, Part 1: Substrate Dependence. http://www.paul-almond.com/Substrate1.pdf.
Minds, Substrate, Measure and Value, Part 2: Extra Information About Substrate Dependence. http://www.paul-almond.com/Substrate2.pdf.
Minds, Substrate, Measure and Value, Part 3: The Problem of Arbitrariness of Interpretation. http://www.paul-almond.com/Substrate3.pdf.
This won’t resolve everything, but should show that the kind of ontology you are talking about is not a “random free for all”.
If you want a way of phrasing this problem which involves the agent being in an attainable state, this may be of some small interest, Alexandros. A few years back I wrote an article discussing a situation with some similiarities with the one in Newcomb’s problem and with an attainable-state agent. While the article doesn’t prove anything really profound in philosophy, it might give a useful context. It is here: http://www.paul-almond.com/GameTheoryWithYourself.htm.