PhilosophyTutor

Karma: 139

PhilosophyTutor 7 May 2014 22:43 UTC
2 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
(EDIT: See below.) I’m afraid that I am now confused. I’m not clear on what you mean by “these traits”, so I don’t know what you think I am being confident about. You seem to think I’m arguing that AIs will converge on a safe design and I don’t remember saying anything remotely resembling that.

EDIT: I think I figured it out on the second or third attempt. I’m not 100% committed to the proposition that if we make an AI and know how we did so that we can definitely make sure it’s fun and friendly, as opposed to fundamentally uncontrollable and unknowable. However it seems virtually certain to me that we will figure out a significant amount about designing AIs to do what we want in the process of developing them. People who subscribe to various “FOOM” theories about AI coming out of nowhere will probably disagree with this as is their right, but I don’t find any of those theories plausible.

I also I hope I didn’t give the impression that I thought it was meaningfully possible to create a God-like AI without understanding how to make AI. It’s conceivable in that such a creation story is not a logical contradiction like a square circle or a colourless green dream sleeping furiously, but that is all. I think it is actually staggeringly unlikely that we will make an AI without either knowing how to make an AI, or knowing how to upload people who can then make an AI and tell use how they did it.

PhilosophyTutor 7 May 2014 11:05 UTC
1 point
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I won’t argue against the claim that we could conceivably create an AI without knowing anything about how to create an AI. It’s trivially true in the same way that we could conceivably turn a monkey loose on a typewriter and get strong AI.

I also agree with you that if we got an AI that way we’d have no idea how to get it to do any one thing rather than another and no reason to trust it.

I don’t currently agree that we could make such an AI using a non-functioning brain model plus “a bit of evolution”. I am open to argument on the topic but currently it seems to me that you might as well say “magic” instead of “evolution” and it would be an equivalent claim.

PhilosophyTutor 3 May 2014 3:03 UTC
2 points
in reply to: shminux’s comment on: Siren worlds and the perils of over-optimised search
A universal measure for anything is a big demand. Mostly we get by with some sort of somewhat-fuzzy “reasonable person” standard, which obviously we can’t fully explicate in neurological terms either yet, but which is much more achievable.

Liberty isn’t a one-dimensional quality either, since for example you might have a country with little real freedom of the press but lots of freedom to own guns, or vice versa.

What you would have to do to develop a measure with significant intersubjective validity is to ask a whole bunch of relevantly educated people what things they consider important freedoms and what incentives they would need to be offered to give them up, to figure out how they weight the various freedoms. This is quite do-able, and in fact we do very similar things when we do QALY analysis of medical interventions to find out how much people value a year of life without sight compared to a year of life with sight (or whatever).

Fundamentally it’s not different to figuring out people’s utility functions, except we are restricting the domain of questioning to liberty issues.

PhilosophyTutor 3 May 2014 0:17 UTC
0 points
in reply to: shminux’s comment on: Siren worlds and the perils of over-optimised search
I tend to think that you don’t need to adopt any particular position on free will to observe that people in North Korea lack freedom from government intervention in their lives, access to communication and information, a genuine plurality of viable life choices and other objectively identifiable things humans value. We could agree for the sake of argument that “free will is an illusion” (for some definitions of free will and illusion) yet still think that people in New Zealand have more liberty than people in North Korea.

I think that you are basically right that the Framing Problem is like the problem of building a longer bridge, or a faster car, in that you are never going to solve the entire class of problem at a stroke so that you can make infinitely long bridges or infinitely fast cars but that you can make meaningful incremental progress over time. I’ve said from the start that capturing the human ability to make philosophical judgments about liberty is a hard problem but I don’t think it is an impossible one—just a lot easier than creating a program that does that and solves all the other problems of strong AI at once.

In the same way that it turns out to be much easier to make a self-driving car than a strong AI, I think we’ll have useful natural-language parsing of terms like “liberty” before we have strong AI.

PhilosophyTutor 2 May 2014 20:37 UTC
−2 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I said earlier in this thread that we can’t do this and that it is a hard problem, but also that I think it’s a sub-problem of strong AI and we won’t have strong AI until long after we’ve solved this problem.

I know that Word of Eliezer is that disciples won’t find it productive to read philosophy, but what you are talking about here has been discussed by analytic philosophers and computer scientists as “the frame problem” since the eighties and it might be worth a read for you. Fodor argued that there are a class of “informationally unencapsulated” problems where you cannot specify in advance what information is and is not relevant to solving the problem, hence really solving them as opposed to coming up with a semi-reliable heuristic is an incredibly difficult problem for AI. Defining liberty or identifying it in the wild seems like it’s an informationally unencapsulated problem in that sense and hence a very hard one, but one which AI has to solve before it can tackle the problems humans tackle.

If I recall correctly Fodor argued in Modules, Frames, Fridgeons, Sleeping Dogs, and the Music of the Spheres that this problem was in fact the heart of the AI problem, but that proposition was loudly raspberried in the literature by computer scientists. You can make up your own mind about that one.

Here’s a link to the Stanford Dictionary of Philosophy page on the subject.

PhilosophyTutor 2 May 2014 20:19 UTC
0 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I didn’t think we needed to put the uploaded philosopher under billions of years of evolutionary pressure. We would put your hypothetical pre-God-like AI in one bin and update it under pressure until it becomes God-like, and then we upload the philosopher separately and use them as a consultant.

(As before I think that the evolutionary landscape is unlikely to allow a smooth upward path from modern primate to God-like AI, but I’m assuming such a path exists for the sake of the argument).

PhilosophyTutor 2 May 2014 0:49 UTC
2 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I think there is insufficient information to answer the question as asked.

If I offer you the choice of a box with $5 in it, or a box with $500 000 in it, and I know that you are close enough to a rational utility-maximiser that you will take the $500 000, then I know what you will choose and I have set up various facts in the world to determine your choice. Yet it does not seem on the face of it as if you are not free.

On the other hand if you are trying to decide between being a plumber or a blogger and I use superhuman AI powers to subtly intervene in your environment to push you into one or the other without your knowledge then I have set up various facts in the world to determine your choice and it does seem like I am impinging on your freedom.

So the answer seems to depend at least on the degree of transparency between A and B in their transactions. Many other factors are almost certainly relevant, but that issue (probably among many) needs to be made clear before the question has a simple answer.

PhilosophyTutor 1 May 2014 11:49 UTC
0 points
in reply to: [deleted]’s comment on: Siren worlds and the perils of over-optimised search
If I was unclear, I was intending that remark to apply to the original hypothetical scenario where we do have a strong AI and are trying to use it to find a critical path to a highly optimal world. In the real world we obviously have no such capability. I will edit my earlier remark for clarity.

PhilosophyTutor 1 May 2014 11:46 UTC
4 points
in reply to: [deleted]’s comment on: Siren worlds and the perils of over-optimised search
The standard LW position (which I think is probably right) is that human brains can be modelled with Turing machines, and if that is so then a Turing machine can in theory do whatever it is we do when we decide that something ls liberty, or pornography.

There is a degree of fuzziness in these words to be sure, but the fact we are having this discussion at all means that we think we understand to some extent what the term means and that we value whatever it is that it refers to. Hence we must in theory be able to get a Turing machine to make the same distinction although it’s of course beyond our current computer science or philosophy to do so.

PhilosophyTutor 1 May 2014 0:16 UTC
−1 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
If you can do that, then you can just find someone who you think understands what we mean by “liberty” (ideally someone with a reasonable familiarity with Kant, Mill, Dworkin and other relevant writers), upload their brain without understanding it, and ask the uploaded brain to judge the matter.

(Off-topic: I suspect that you cannot actually get a markedly superhuman AI that way, because the human brain could well be at or near a peak in the evolutionary landscape so that there is no evolutionary pathway from a current human brain to a vastly superhuman brain. Nothing I am aware of in the laws of physics or biology says that there must be any such pathway, and since evolution is purposeless it would be an amazing lucky break if it turned out that we were on the slope of the highest peak there is, and that the peak extends to God-like heights. That would be like if we put evolutionary pressure on a cheetah and discovered that if we do that we can evolve a cheetah that runs at a significant fraction of c.

However I believe my argument still works even if I accept for the sake of argument that we are on such a peak in the evolutionary landscape, and that creating God-like AI is just a matter of running a simulated human brain under evolutionary pressure for a few billion simulated years. If we have that capability then we must also be able to run a simulated philosopher who knows what “liberty” refers to).

EDIT: Downvoting this without explaining why you disagree doesn’t help me understand why you disagree.

PhilosophyTutor 1 May 2014 0:07 UTC
0 points
in reply to: CCC’s comment on: Siren worlds and the perils of over-optimised search

Why? Just because the problem is less complicated, does not mean it will be solved first. A more complicated problem can be solved before a less complicated problem, especially if there is more known about it.

To clarify, it seems to me that modelling hairyfigment’s ability to decide whether people have liberty is not only simpler than modelling hairyfigment’s whole brain, but that it is also a subset of that problem. It does seem to me that you have to solve all subsets of Problem B before you can be said to have solved Problem B, hence you have to have solved the liberty-assessing problem if you have solved the strong AI problem, hence it makes no sense to postulate a world where you have a strong AI but can’t explain liberty to it.

PhilosophyTutor 1 May 2014 0:02 UTC
1 point
in reply to: hairyfigment’s comment on: Siren worlds and the perils of over-optimised search
We have identified the point on which we differ, which is excellent progress. I used fictional worlds as examples, but would it solve the problem if I used North Korea and New Zealand as examples instead, or the world in 1814 and the world in 2014? Those worlds or nations were not created to be transparent to human examination but I believe you do have the faculty to distinguish between them.

I don’t see how this is harder than getting an AI to handle any other context-dependant, natural language descriptor, like “cold” or “heavy”. “Cold” does not have a single, unitary definition in physics but it is not that hard a problem to figure out when you should say “that drink is cold” or “that pool is cold” or “that liquid hydrogen is cold”. Children manage it and they are not vastly superhuman artificial intelligences.

PhilosophyTutor 30 Apr 2014 12:10 UTC
3 points
in reply to: hairyfigment’s comment on: Siren worlds and the perils of over-optimised search
I’ll try to lay out my reasoning in clear steps, and perhaps you will be able to tell me where we differ exactly.
1. Hairyfigment is capable of reading Orwell’s 1984, and Banks’ Culture novels, and identifying that the people in the hypothetical 1984 world have less liberty than the people in the hypothetical Culture world.
2. This task does not require the full capabilities of hairyfigment’s brain, in fact it requires substantially less.
3. A program that does A+B has to be more complicated than a program that does A alone, where A and B are two different, significant sets of problems to solve. (EDIT: If these programs are efficiently written)
4. Given 1-3, a program that can emulate hairyfigment’s liberty-distinguishing faculty can be much, much less complicated than a program that can do that plus everything else hairyfigment’s brain can do.
5. If we can simulate a complete human brain that is the same as having solved the strong AI problem.
6. A program that can do everything hairyfigment’s brain can do is a program that simulates a complete human brain.
7. Given 4-6 it is much less complicated to emulate hairyfigment’s liberty-distinguishing faculty than to solve the strong AI problem.
8. Given 7, it is unreasonable to postulate a world where we have solved the strong AI problem, in spades, so much so we have a vastly superhuman AI, but we still haven’t solved the hairyfigment’s liberty-distinguishing faculty problem.

PhilosophyTutor 30 Apr 2014 8:07 UTC
4 points
in reply to: hairyfigment’s comment on: Siren worlds and the perils of over-optimised search
I really am. I think a human brain could rule out superficially attractive dystopias and also do many, many other things as well. If you think you personally could distinguish between a utopia and a superficially attractive dystopia given enough relevant information (and logically you must think so, because you are using them as different terms) then it must be the case that a subset of your brain can perform that task, because it doesn’t take the full capabilities of your brain to carry out that operation.

I think this subtopic is unproductive however, for reasons already stated. I don’t think there is any possible world where we cannot achieve a tiny, partial solution to the strong AI problem (codifying “liberty”, and similar terms) but we can achieve a full-blown, transcendentally superhuman AI. The first problem is trivial compared to the second. It’s not a trivial problem, by any means, it’s a very hard problem that I don’t see being overcome in the next few decades, but it’s trivial compared to the problem of strong AI which is in turn trivial compared to the problem of vastly superhuman AI. I think Stuart_Armstrong is swallowing a whale and then straining at a gnat.

PhilosophyTutor 30 Apr 2014 6:28 UTC
2 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I could be wrong but I believe that this argument relies on an inconsistent assumption, where we assume we have solved the problem of creating an infinitely powerful AI, but we have not solved the problem of operationally defining commonplace English words which hundreds of millions of people successfully understand in such a way that a computer can perform operations using them.

It seems to me that the strong AI problem is many orders of magnitude more difficult than the problem of rigorously defining terms like “liberty”. I imagine that a relatively small part of the processing power of one human brain is all that is needed to perform operations on terms like “liberty” or “paternalism” and engage in meaningful use of them so it is a much, much smaller problem than the problem of creating even a single human-level AI, let alone a vastly superhuman AI.

If in our imaginary scenario we can’t even define “liberty” in such a way that a computer can use the term, it doesn’t seem very likely that we can build any kind of AI at all.

PhilosophyTutor 29 Apr 2014 21:40 UTC
5 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search

The strong AI problem is much easier to solve than the problem of motivating an AI to respect liberty. For instance, the first one can be brute forced (eg AIXItl with vast resources), the second one can’t.

I don’t believe that strong AI is going to be as simple to brute force as a lot of LessWrongers believe, personally, but if you can brute force strong AI then you can just get it to run a neuron-by-neuron simulation of the brain of a reasonably intelligent first year philosophy student who understands the concept of liberty and tell the AI not to take actions which the simulated brain thinks offend against liberty.

That is assuming that in this hypothetical future scenario where we have a strong AI we are capable of programming that strong AI to do any one thing instead of another, but if we cannot do that then the entire discussion seems to me to be moot.

PhilosophyTutor 29 Apr 2014 11:34 UTC
0 points
in reply to: drnickbone’s comment on: Siren worlds and the perils of over-optimised search
I think Asimov did this first with his Multivac stories, although rather than promptly destroy itself Multivac executed a long-term plan to phase itself out.

PhilosophyTutor 29 Apr 2014 11:29 UTC
5 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search

Precisely and exactly! That’s the whole of the problem—optimising for one thing (appearance) results in the loss of other things we value.

This just isn’t always so. If you instruct an AI to optimise a car for speed, efficiency and durability but forget to specify that it has to be aerodynamic, you aren’t going to get a car shaped like a brick. You can’t optimise for speed and efficiency without optimising for aerodynamics too. In the same way it seems highly unlikely to me that you could optimise a society for freedom, education, just distribution of wealth, sexual equality and so on without creating something pretty close to optimal in terms of unwanted pregnancies, crime and other important axes.

Even if it’s possible to do this, it seems like something which would require extra work and resources to achieve. A magical genie AI might be able to make you a super-efficient brick-shaped car by using Sufficiently Advanced Technology indistinguishable from magic but even for that genie it would have to be more work than making an equally optimal car by the defined parameters that wasn’t a silly shape. In the same way an effectively God-like hypothetical AI might be able to make a siren world that optimised for everything except crime and create a world perfect in every way except that it was rife with crime but it seems like it would be more work, not less.

Next challenge: define liberty in code. This seems extraordinarily difficult.

I think if we can assume we have solved the strong AI problem, we can assume we have solved the much lesser problem of explaining liberty to an AI.

So we do agree that there are problem with an all-powerful genie?

We’ve got a problem with your assumptions about all-powerful genies, I think, because I think your argument relies on the genie being so ultimately all-powerful that it is exactly as easy for the genie to make an optimal brick-shaped car or an optimal car made out of tissue paper and post-it notes as it is for the genie to make an optimal proper car. I don’t think that genie can exist in any remotely plausible universe.

If it’s not all-powerful to that extreme then it’s still going to be easier for the genie to make a society optimised (or close to it) across all the important axes at once than one optimised across all the ones we think to specify while tanking all the rest. So for any reasonable genie I still think market worlds don’t make sense as a concept. Siren worlds, sure. Market worlds, not so much, because the things we value are deeply interconnected and you can’t just arbitrarily dump-stat some while efficiently optimising all the rest.

PhilosophyTutor 28 Apr 2014 21:03 UTC
5 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I think this and the “finite resources therefore tradeoffs” argument both fail to take seriously the interconnectedness of the optimisation axes which we as humans care about.

They assume that every possible aspect of society is an independent slider which a sufficiently advanced AI can position at will, even though this society is still going to be made up of humans, will have to be brought about by or with the cooperation of humans and will take time to bring about. These all place constraints on what is possible because the laws of physics and human nature aren’t infinitely malleable.

I don’t think discreet but total control over a world is compatible with things like liberty, which seem like obvious qualities to specify in an optimal world we are building an AI to search for.

I think what we might be running in to here is less of an AI problem and more of a problem with the model of AI as an all-powerful genie capable of absolutely anything with no constraints whatsoever.

PhilosophyTutor 28 Apr 2014 11:21 UTC
2 points
in reply to: Stuart_Armstrong’s comment on: Siren worlds and the perils of over-optimised search
I don’t think you have highlighted a fundamental problem since we can just specify that we mean a low percentage of conceptions being deliberately aborted in liberal societies where birth control and abortion are freely available to all at will.

My point, though, is that I don’t think it is very plausible that “marketing worlds” will organically arise where there are no humans, or no conception, but which tick all the other boxes we might think to specify in our attempts to describe an ideal world. I don’t see how there being no conception or no humans could possibly be a necessary trade-off with things like wealth, liberty, rationality, sustainability, education, happiness, the satisfaction of rational and well-informed preferences and so forth.

Of course a sufficiently God-like malevolent AI could presumably find some way of gaming any finite list we give it, since there are probably an unbounded number of ways of bringing about horrible worlds, so this isn’t a problem with the idea of siren worlds. I just don’t find the idea of market worlds very plausible because so many of the things we value are fundamentally interconnected.