Toby_Ord2

Karma: 107

Toby_Ord2 11 Jan 2009 17:32 UTC
2 points
on: Continuous Improvement
the value of this memory card, was worth more than the rest of the entire observable universe minus the card

I doubt this would be true. I think the value of the card would actually be close to zero (though I’m not completely sure). It does let one solve the halting problem up to 10,000 states, but it does so in time and space complexity O(busy_beaver(n)). In other words, using the entire observable universe as computing power and the card as an oracle, you might be able to solve the halting problem for 7 state machines or so. Not that good… The same goes for having the first 10,000 bits of Omega. What you really want are the bits of Tau, which directly encode whether the nth machine halts. Sure you need exponentially more of them, but your computation is then much faster.

Toby_Ord2 4 Jan 2009 22:09 UTC
0 points
on: Free to Optimize
OK. That makes more sense then. I’m not sure why you call it ‘Fun Theory’ though. It sounds like you intend it to be a theory of ‘the good life’, but a non-hedonistic one. Strangely it is one where people having ‘fun’ in the ordinary sense is not what matters, despite the name of the theory.

This is a moral theory about what should be fun

I don’t think that can be right. You are not saying that there is a moral imperative for certain things to be fun, or to not be fun, as that doesn’t really make sense (at least I can’t make sense of it). You are instead saying that certain conditions are bad, even when the person is having fun (in the ordinary sense). Maybe you are saying that what is good for someone mostly maps to their fun, but with several key exceptions (which the theory then lists).

In any event, I agree with Z.M. Davis that you should capitalize your ‘Fun’ when you are using it in a technical sense, and explaining the sense in more detail or using a different word altogether might also help.

Toby_Ord2 4 Jan 2009 19:32 UTC
2 points
on: Free to Optimize
Eliezer,

Are you saying that one’s brain state can be identical in two different scenarios but that you are having a different amount of fun in each? If so, I’m not sure you are talking about what most people call fun (ie a property of your experiences). If not, then what quantity are you talking about in this post where you have less of it if certain counterfactuals are true?

Toby_Ord2 4 Nov 2008 14:20 UTC
1 point
on: Complexity and Intelligence
I would drop dead of shock

Eliezer, just as it was interesting to ask what probability estimate ‘Nuts!’ amounted to, I think it would be very useful for the forum of Overcoming Bias to ask what your implicit probability estimate for a 500 state TM being able to solve the halting problem for all TMs of up to 50 states.

I imagine that ‘I would drop dead of shock’ was intended to convey a probability of less than 1 in 10,000, or maybe 1 in 1,000,000?

Toby_Ord2 29 Oct 2008 22:07 UTC
0 points
on: Economic Definition of Intelligence?
Sorry, I didn’t see that you had answered most of this question in the other thread where I first asked it.

Toby, if you were too dumb to see the closed-form solution to problem 1, it might take an intense effort to tweak the bit on each occasion, or perhaps you might have trouble turning the global criterion of total success or failure into a local bit-fixer; now imagine that you are also a mind that finds it very easy to sing MP3s...

The reason you think one problem is simple is that you perceive a solution in closed form; you can imagine a short program, much shorter than 10 million bits, that solves it, and the work of inventing this program was done in your mind without apparent effort. So this problem is very trivial on the meta-level because the program that solves it optimally appears very quickly in the ordering of possible programs and is moreover prominent in that ordering relative to our instinctive transformations of the problem specification.

But if you were trying random solutions and the solution tester was a black box, then the alternating-bits problem would indeed be harder—so you can’t be measuring the raw difficulty of optimization if you say that one is easier than the other.

This is why I say that the human notion of “impressiveness” is best constructed out of a more primitive notion of “optimization”.

We also do, legitimately, find it more natural to talk about “optimized” performance on multiple problems than on a single problem—if we’re talking about just a single problem, then it may not compress the message much to say “This is the goal” rather than just “This is the output.”

I take it then that you agree that (1) is a problem of 9,999,999 bits and that the travelling salesman version is as well. Could you take these things and generate an example which doesn’t just give ‘optimization power’, but ‘intelligence’ or maybe just ‘intelligence-without-adjusting-for-resources-spent’. You say over a set of problem domains, but presumably not over all of them given the no-free-lunch theorems. Any example, or is this vague?

Toby_Ord2 29 Oct 2008 22:01 UTC
5 points
on: Economic Definition of Intelligence?
Eliezer,

I’m afraid that I’m not sure precisely what your measure is, and I think this is because you have given zero precise examples: even of its subcomponents. For example, here are two optimization problems:

1) You have to output 10 million bits. The goal is to output them so that no two consecutive bits are different.

2) You have to output 10 million bits. The goal is to output them so that when interpreted as an MP3 file, they would make a nice sounding song.

Now, the solution space for (1) consists of two possibilities (all 1s, all 0s) out of 2^10000000, for a total of 9,999,999 bits. The solution space for (2) is millions of times wider, leading to fewer bits. However, intuitively, (2) is a much harder problem and things that optimized (2) are actually doing more of the work of intelligence, after all (1) can be achieved in a few lines of code and very little time or space, while (2) takes much more of these resources.

(2) is a pretty complex problem, but can you give some specifics for (1)? Is it exactly 9,999,999 bits? If so, is this the ‘optimization power’? Is this a function of the size of the solution space and the size of the problem space only? If there was another program attempting to produce a sequence of 100 million bits coding some complex solution to a large travelling salesman problem, such that only two bitstrings suffice, would this have the same amount of optimization power?, or is it a function of the solution space itself and not just its size?

Without even a single simple example, it is impossible to narrow down your answer enough to properly critique it. So far I see it as no more precise than Legg and Hutter’s definition.

Toby_Ord2 28 Oct 2008 17:06 UTC
0 points
on: Measuring Optimization Power
I agree with David’s points about the roughness of the search space being a crucial factor in a meaningful definition of optimization power.

Toby_Ord2 28 Oct 2008 17:04 UTC
1 point
on: Measuring Optimization Power
I’m not sure that I get this. Perhaps I understand the maths, but not the point of it. Here are two optimization problems:

1) You have to output 10 million bits. The goal is to output them so that no two consecutive bits are different.

2) You have to output 10 million bits. The goal is to output them so that when interpreted as an MP3 file, they would make a nice sounding song.

Now, the solution space for (1) consists of two possibilities (all 1s, all 0s) out of 2^10000000, for a total of 9,999,999 bits. The solution space for (2) is millions of times wider, leading to fewer bits. However, intuitively, (2) is a much harder problem and things that optimized (2) are actually doing more of the work of intelligence, after all (1) can be achieved in a few lines of code and very little time or space, while (2) takes much more of these resources.

Toby_Ord2 22 Oct 2008 11:22 UTC
0 points
on: Ethics Notes
But if you say “Shut up and do what seems impossible!”, then that, to me, sounds like dispelling part of the essential message—that what seems impossible doesn’t look like it “seems impossible”, it just looks impossible.

“Shut up and do what seems impossible!” is the literally correct message. The other one is the exaggerated form. Sometimes exaggeration is a good rhetorical device, but it does turn off some serious readers.

“Don’t do it, even if it seems right” sounds merely clever by comparison

This was my point. This advice is useful and clever, though not profound. This literal presentation is both more clear in what it is saying and clear that it is not profound. I would have thought that the enterprise of creating statements that sound more profound than they are is not a very attractive one for rationalists. Memorable statements are certainly a good thing, but making them literally false and spuriously paradoxical does not seem worth it. This isn’t playing fair. Any statement can be turned into a pseudo-profundity with these methods: witness many teachings of cults throughout the ages. I think these are the methods of what you have called ‘Dark Side Epistemology’.

Toby_Ord2 22 Oct 2008 9:47 UTC
3 points
on: Prices or Bindings?
Eliezer,

Crossman and Crowley make very good points above, delineating three possible types of justification for some of the things you say:

1) Don’t turn him in because the negative effects of the undermining of the institution will outweigh the benefits

2) Don’t turn him in because [some non-consequentialist reason on non-consequentialist grounds]

3) Don’t turn him in because you will have rationally/consequentialistly tied yourself to the mast making it impossible to turn him in to achieve greater benefits.

(1) and (3) are classic pieces of consequentialism, the first dating back at least to Mill. If your reason is like those, then you are probably a consequentialist and there is no need to reinvent the wheel: I can provide some references for you. If you support (2), perhaps on some kind of Newcomb’s problem grounds, then this deserves a clear explanation. Why, on account of a tricky paradoxical situation that may not even be possible, will you predictably start choosing to make things worse in situations that are not Newcomb situations? Unless you are explicit about your beliefs, we can’t help debug them effectively, and you then can’t hold them with confidence for they won’t have undergone peer scrutiny. [The same still goes for your meta-ethical claims].

Toby_Ord2 21 Oct 2008 10:33 UTC
4 points
on: Ethical Injunctions
You should never, ever murder an innocent person who’s helped you, even if it’s the right thing to do

Shut up and do the impossible!

As written, both these statements are conceptually confused. I understand that you didn’t actually mean either of them literally, but I would advise against trading on such deep-sounding conceptual confusions.

You should never, ever do X, even if if you are exceedingly confident that it is the right thing to do

This sounds less profound, but will actually be true for some value of X, unlike the first sentence or its derivatives. It sounds as profound as it is, and no more. I believe this is the right standard.

Toby_Ord2 1 Sep 2008 18:32 UTC
2 points
on: Brief Break
Eli:

It is certainly similar to those problems, but slightly different. For example, justifying Occam’s Razor requires a bit more than we need here. In our case, we are just looking for a canonical complexity measure for finite strings. For Occam’s Razor we also need to show that we have reason to prefer theories expressible by simpler strings to those specified by more complex strings. As an example, we already have such a canonical complexity measure for infinite strings. It is not perfect, as you might want some complexity measure defined with o-machines instead, or with finite state automata or whatever. These would give different complexity measures, but at least the Turing machine level one marks out a basic type of complexity, rather than an infinite set of complexity measures, as for finite strings.

Shane:

Where are you going to put all this?

Is that a physical question? If so, it is just complexity-relative-to-physics or to be more precise: complexity-relative-to-physical-size. If you mean that it seems more complex: of course it does to us, but we have no canonical way of showing that, as opposed to measures based on its rival machines from the same computability class (which is just begging the question). As I said, there might be a canonical complexity measure for finite strings, but if so, this hasn’t been proven yet. I don’t know what the upshot of all this is, and in many practical cases I’m sure the stuff I’m saying can be safely put aside, but it is worth being aware of it, particularly when trying to get appropriately general and theoretical results (such as the AIXI stuff). If we talk about AIXI(M) where AIXI is a function of machine M, and call my pathological machine P, then AIXI(TM) looks pretty much the same as AIXI(LambdaCalculus) and AIXI(Java) and every other sensible language we use, but it looks completely different to AIXI(P) until we start looking at strings of order 3^^^3. Whether that is a problem depends upon the questions being asked.

Toby_Ord2 1 Sep 2008 14:05 UTC
2 points
on: Brief Break
Shane:

Why not the standard approach of using Shannon’s state x symbol complexity for Turing machines?

Why choose a Turing machine? They are clearly not a canonical mathematical entity, just a historical artifact. Their level of power is a canonical mathematical entity, but there are many Turing-equivalent models of computation. This just gets us simplicity relative to Turing machines where what we wanted was simplicity simpliciter (i.e. absolute simplicity). If someone came to you with a seemingly bizarre Turing-complete model, where the shortest program for successor was 3^^^3 bits and all the short programs were reserved for things that look crazy to us, how can you show him that Turing machines are the more appropriate model? Of course, it is obvious to us that Turing machines are in some sense a better judge of the notion of simplicity that we want, but can we show this mathematically? If we can, it hasn’t yet been done. Turing machines might look simple to Turing machines (and human brains) but this new model might look simple to itself (e.g. has a 1 bit universal machine etc.).

It looks like we have to settle for a non-canonical concept of simplicity relative to human intuitions or simplicity relative to physics or the like. I think this is a deep point, which is particularly puzzling and not sufficiently acknowledged in Kolmolgorov complexity circles. It feels like there must be some important mathematically canonical measure of complexity of finite strings, just like there is for infinite strings, but all we have are ‘complexity-relative-to-X’ and perhaps this is all there is.

Toby_Ord2 1 Sep 2008 10:33 UTC
0 points
on: Brief Break
Shane:

That’s why a tiny reference machine is used.

I think that Tim is pointing out that there is no available mathematical measure for the ‘tinyness’ of this machine which is not circular. You seem to be saying that the machine looks simple to most people and that all other machines which people class as simple could be simulated on this machine within a few hundred bits. This has two problems. Firstly, it is not provable that all other machines which we class as similarly simple will be simulated within a few hundred bits as it is an empirical question which other machines people find simple. I’ll grant that we can be reasonably confident though. The second problem is that we haven’t defined any canonical mathematical measure of simplicity, just a measure of ‘simplicity relative to the empirical facts about what humans find simple’. Perhaps we could use physics instead of humans and look at physically small Turing machines and then have ‘simplicity relative to the empirical facts about what can be done in small volumes’. These are no doubt interesting, but are still concepts of relative simplicity, not a canonical absolute simplicity/complexity. No such measure has been discovered, and perhaps there can be no such measure. We can contrast this with complexity of infinite strings, where there is a convergence between all base machines and thus an absolute measure of simplicity. The problem is that we are now looking for something to deal with finite strings, not infinite ones.

Toby_Ord2 13 Aug 2008 13:17 UTC
5 points
on: Abstracted Idealized Dynamics
Great! Now I can see several points where I disagree or would like more information.

1) Is X really asserting that Y shares his ultimate moral framework (i.e. that they would converge given time and arguments etc)?

If Y is a psychopath murderer who will simply never accept that he shouldn’t kill, can I still judge that Y should refrain from killing? On the current form, to do so would involve asserting that we share a framework, but even people who know this to be false can judge that he shouldn’t kill, can’t they?

2) I don’t know what it means to be the solution to a problem. You say:

‘I should Z’ means that Z answers the question, “What will save my people? How can we all have more fun? How can we get more control over our own lives? What’s the funniest jokes we can tell? …”

Suppose Z is the act of saying “no”. How does this answer the question (or ‘solve the problem’)? Suppose it leads you to have a bit less fun and others to have a bit more fun and generally has positive effects on some parts of the question and negative on others. How are these integrated? As you phrased it, it is clearly not a unified question and I don’t know what makes one act rather than another an answer to a list of questions (when presumably it doesn’t satisfy each one in the list). Is there some complex and not consciously known weighting of the terms? I thought you denied that earlier in the series. This part seems very non-algorithmic at the moment.

3) The interpretation says ‘implicitly defined by the machinery … which they both use to make desirability judgments’?

What if there is not such machinery that they both use? I thought only X’s machinery counted here as X is the judger.

4) You will have to say more about ‘implicitly defined by the machinery … use[d] to make desirability judgments’. This is really vague. I know you have said more on this, but never in very precise terms, just by analogy.

5) Is the problem W meant to be the endpoint of thought (i.e. the problem that would be arrived at), or is it meant to be the current idea which involves requests for self modification (e.g. ‘Save a lot of lives, promote happiness, and factor in whatever things I have not thought of but could be convinced of.’) It is not clear from the current statement (or indeed your previous posts), but would be made clear by a solution to (4).

Toby_Ord2 12 Aug 2008 20:54 UTC
0 points
on: Abstracted Idealized Dynamics
Eliezer,

I didn’t mean that most philosophy papers I read have lots of mathematical symbols (they typically don’t), and I agree with you that over-formalization can occur sometimes (though it is probably less common in philosophy than under-formalization). What I meant is the practice of clear and concise statements of the main points and attendant qualifications in the kind of structured English that good philosophers use. For example, I gave the following as a guess at what you might be meaning:

When X judges that Y should Z, X is judging that were she fully informed, she would want Y to Z

This allows X to be incorrect in her judgments (if she wouldn’t want Y to Z when given full information). It allows for others to try to persuade X that her judgment is incorrect (it preserves a role for moral argument). It reduces ‘should’ to mere want (which is arguably simpler). It is, however, a conception of should that is judger-dependent: it could be the case that X correctly judges that Y should Z, while W correctly judges that Y should not Z.

The first line was a fairly clear and concise statement of a meta-ethical position (which you said you don’t share, and nor do I for that matter). The next few sentences describe some of its nice features as well as a downside. There is very little technical language—just ‘judge’, ‘fully informed’ and ‘want’. In the previous comment I gave a sentence or two saying what was meant by ‘fully informed’ and if challenged I could have described the other terms. Given that you think it is incorrect, could you perhaps fix it, providing a similar short piece of text that describes your view with a couple of terms that can bear the brunt of further questioning and elaboration.

Toby_Ord2 12 Aug 2008 10:37 UTC
0 points
on: Abstracted Idealized Dynamics
Eliezer,

I agree with most of the distinctions and analogies that you have been pointing out, but I still doubt that I agree with your overall position. No-one here can know whether they agree with your position because it is very much underdetermined by your posts. I can have a go at formulating what I see as the strongest objections to your position if you clearly annunciate it in one place. Oddly enough, the philosophy articles that I read tend to be much more technically precise than your posts. I don’t mean that your couldn’t write more technically precise posts on metaethics, just that I would like you to.

In the same way as scientific theories need to be clear enough to allow concrete prediction and potential falsification, so philosophical theories need to be clear enough that others can use them without any knowledge of their author to make new claims about their subject matter. Many people here may feel that you have made many telling points (which you have), but I doubt that they understand your theory in the sense that they could apply it in wide range of situations where it is applicable. I would love a short post consisting of at most a paragraph of introduction, then a bi-conditional linking a person’s judgement about what another person should do in a given situation to some naturalistic facts and then a paragraph or two helping resolve any ambiguities. Then others can actually argue against it and absence of argument could start to provide some evidence in its favour (though of course, surviving the criticisms of a few grad-student philosophers would still not be all that much evidence).

Toby_Ord2 8 Aug 2008 12:04 UTC
5 points
on: Morality as Fixed Computation
Eliezer,

Sorry for not being more precise. I was actually asking what a given person’s Q_P is, put in terms that we have already defined. You give a partial example of such a question, but it is not enough for me to tell what metaethical theory you are expressing. For example, suppose Mary currently values her own pleasure and nothing else, but that were she exposed to certain arguments she would come to value everyone’s pleasure (in particular, the sum of everyone’s pleasure) and that no other arguments would ever lead her to value anything else. This is obviously unrealistic, but I’m trying to determine what you mean via a simple example. Would Q_Mary be ‘What maximizes Mary’s pleasure?’ or ‘What maximizes the sum of pleasure?’ or would it be something else? On my attempted summary, Q_Mary would be the second of these questions as that is what she would want if she knew all relevant arguments. Also, does it matter whether we suppose that Mary is open to change to her original values or if she is strongly opposed to change to her original values?

(Items marked in bold have to be morally evaluated.)

I don’t think so. For example, when I said ‘incorrect’ I meant ‘made a judgement which was false’. When I said ‘best’ arguments, I didn’t mean the morally superior arguments, just the ones that are most convincing (just as the ‘best available scientific theory’ is not a moral claim). Feel free to replace that with something like ‘if she had access to all relevant arguments’, or ‘if there exists an argument which would convince her’ or the like. There are many ways this could be made precise, but it is not my task to do so: I want you to do so, so that I can better see and reply to your position.

Regarding the comment about assessing future Q_Ps from the standpoint of old ones, I still don’t see a precise answer here. For example, if Q_P,T1 approves of Q_P,T2 which approves of Q_P,T3 but Q_P,T1 doesn’t approve of Q_P,T3, then what are we to say? Did two good changes make a bad change?

Toby_Ord2 8 Aug 2008 10:25 UTC
3 points
on: Morality as Fixed Computation
Thanks for responding to my summary attempt. I agree with Robin that it is important to be able to clearly and succinctly express your main position, as only then can it be subject to proper criticism to see how well it holds up. In one way, I’m glad that you didn’t like my attempted summary as I think the position therein is false, but it does mean that we should keep looking for a neat summary. You currently have:

‘I should X’ means that X answers the question, “What will save my people? How can we all have more fun? How can we get more control over our own lives? What’s the funniest jokes we can tell? …”

But I’m not clear where the particular question is supposed to come from. I understand that you are trying to make it a fixed question in order to avoid deliberate preference change or self-fulling questions. So lets say that for each person P, there is a specific question Q_P such that:

For a person P, ‘I should X’, means that X answers the question Q_P.

Now how is Q_P generated? Is it what P would want were she given access to all the best empirical and moral arguments (what I called being fully informed)? If so, do we have to time index the judgment as well? i.e. if P’s preferences change at some late time T1, then did the person mean something different by ‘I should X’ before and after T1 , or was the person just incorrect at one of those times? What if the change is just through acquiring better information (empirical or moral)?

Toby_Ord2 30 Jul 2008 23:53 UTC
0 points
on: The Meaning of Right
To cover cases where people are making judgments about what others should do, I could also extend this summary in a slightly more cumbersome way:

When X judges that Y should Z, X is judging that were she fully informed, she would want Y to Z

This allows X to be incorrect in her judgments (if she wouldn’t want Y to Z when given full information). It allows for others to try to persuade X that her judgment is incorrect (it preserves a role for moral argument). It reduces ‘should’ to mere want (which is arguably simpler). It is, however, a conception of should that is judger-dependent: it could be the case that X correctly judges that Y should Z, while W correctly judges that Y should not Z.