Irgy

Karma: 231

Irgy 2 Nov 2011 6:31 UTC
3 points
on: The Least Convenient Possible World
This might be better placed somewhere else, but I just thought I’d comment on Pascal’s Wager here. To me both the convenient and inconvenient resolutions of Pascal’s Wager given above are quite unsatisfactory.

To me, the resolution of this wager comes from the concept of sets of measure zero. The set of possible realities in which belief in any given God is infinitely beneficial is an infinite set, but it is nonetheless like Cantor Dust in the space of possible explanations of reality. The existence of sets of measure zero explains why it is reasonable to assign the value zero to the probability of something which is not literally impossible. To me, the only true resolutions to this paradox are to either convert, dispute the use of infinity in the utility (which I would also do, although I’m not yet completely convinced either way), or accept that not just is zero an acceptable probability for something that’s not literally impossible but that it’s also the correct value to assign to the probability of an infinitely vindictive God. Everything else is just convenience, egotism or missing the point. Cancelling infinities against other (possibly negative) infinities is simply bad maths, refusing to change your beliefs on the basis of utility rather than evidence is simply not acting in your own best interest and dismissing it based on contradictions in any particular religion or on the existence of other religions is as this article says simply assumed convenience.

In this case, your least convenient world kind of misses the point then. As soon as Omega tells me there’s a non-zero probability of Catholicism being correct (whether Omega has actually told me that or not is not entirely clear mind you) then sure, I’m converting. But this is such a substantial change to reality that I would say it’s essentially removed the paradox. The principle is fine though, I guess my point is just that the least convenient world is not a fixed thing but relative to a particular argument.

It’s interesting to me that the mathematics to resolve the paradox didn’t even exist at the time it was raised. Most people knew there was a problem with it, but at the time (and even today for most people’s level of maths understanding) they had simply no way of expressing it correctly. To me this is actually some justification for a little bit of inertia of beliefs—just because you can’t refute an argument doesn’t mean it’s correct, and your intuition can often tell you something’s wrong before you know what it is. It just needs to be balanced against the many situations where intuition is demonstrably misleading. Innertia and an immovable object are not the same thing.

Irgy 16 Nov 2011 6:41 UTC
8 points
on: Welcome to Less Wrong! (2010-2011)
Hi all.

I’m 30, live in Sydney and work on image processing. I also have a wife and two beautiful daughters, currently nine months and two and a half years old.

I have a strong background in pure maths and an ongoing interest in philosophy. I’ve been a rationalist since before I even knew what one was. Discovering ET Jaynes’ “Probability Theory” was the closest thing I’ll probably ever have to a religious revelation.

I finally wrote down a large explanation of some quite fundamental philosophy I’d had in my head for quite a while and sent it to a couple of friends to get their opinion on it. This prompted one of them to point me here. Since then I’ve read quite a bit, although far from everything, and am enjoying almost every bit of it. I look forward to posting those very thoughts here some time soon, as they appear to still be both novel and consistent with the views here.

I thoroughly enjoy a good forum debate, and have a fairly high opinion (and at least some evidence to back it up) of my ability to think logically and write a well structured (if sometimes overly wordy) argument. Which of course doesn’t mean I’m always right, and, as a good rationalist should, there’s nothing I like more than having my argument torn to shreds by a superior one. I look forward to it happening in the near future.

Irgy 16 Nov 2011 20:46 UTC
−1 points
on: Nonperson Predicates
I think you’re solving the wrong problem. Before you worry about the ethics of super-intelligent AIs creating and deleting human simulations at will, you need to worry about the ethics of humans creating and destroying human+ intelligent AIs at will. To me it’s an amazing display of human-cetrism to only worry about the problem when it’s flipped right back around in the much more distant future.

I realise this doesn’t directly help you solve the problem, but maybe it will give you a different persepective.

Irgy 17 Nov 2011 1:12 UTC
−5 points
in reply to: wedrifid’s comment on: Nonperson Predicates
That’s fine for the most part, but in that case do you really feel that same empathy for these proposed simulations? If all you care about is humans maybe you shouldn’t care about these simulations being killed anyway. They’re less like us than animals, they have no flesh and weren’t born of a mother, why do you care about them just because they make a false imitation of our thoughts?

More importantly though I wasn’t talking about human-centrism as a moral issue but a logical one. Racism is bad because it makes us form groups and mistreat people that are different from us. Racism is stupid on the other hand because it makes us inclined to think people of a different race are more different from ourselves than turns out to actually be the case. Similarly it’s the logical not the moral errors of human-centrism that are really relevant to the discussion. If there’s an ethical issue with killing simulations there’s an ethical issue with killing AIs. Resolve one and you can probably resolve the other. Whether you care or not about either problem is kind of beside the point.

That I also don’t morally support human-centrism is also kind of beside the point.

Irgy 17 Nov 2011 5:17 UTC
0 points
in reply to: Irgy’s comment on: Nonperson Predicates
Ok, forget the poor analogy with racism, why racism is bad is a whole separate issue that I had no intention to get into. Let me try and just explain my point better.

Human-centrism is a bias in thinking which makes us assume things like “The earth is the centre of the universe”, “Only humans have consciousness” and “Morality extends to things approximately as far as they seem like humans”. I personally think it is only through this bias that we would worry about the possible future murder of human simulations before we worry about the possible future murder of the AIs intelligent enough to simulate a human in the first place

Human-centrism as fighting for our tribe and choosing not to respect the rights of AIs is a different issue. Choosing not to respect the rights of AIs is different from failing to appreciate the potential existence of those rights.

Justification Through Pragmatism

Irgy17 Nov 2011 5:39 UTC

−2 points

12 comments4 min readLW link

Irgy 17 Nov 2011 6:41 UTC
2 points
in reply to: Matt_Simpson’s comment on: Justification Through Pragmatism
You say that as if it conflicts with what I’ve written, but it’s exactly my point. As soon as you accept that any of your thoughts are anything less than completely insane, then you’ve already accepted that you have some basic capacity for reason, and therefore accepted the assumption I describe. The traction you then describe getting is exactly the traction I describe as gained by accepting this assumption.

You can certainly consider insanity on a scale from 0 to 1, but in that case this assumption is just that not all of your thoughts are equal to 1 on that scale. As such it’s entirely a yes/no question, not a matter of degree.

Irgy 17 Nov 2011 20:40 UTC
2 points
in reply to: ArisKatsaris’s comment on: Nonperson Predicates
Can’t I use the word “rights” without losing my status as a consquentialist? I simply use the concept of a “being with a right to live” as a shortening for “a being for which murdering would, in the majority of circumstances and all else being equal, be very likely to be a poor moral choice”. You can respect the rights of something without holding a deontological view that rights are somehow the fundamental definition of morality.

Irgy 17 Nov 2011 20:44 UTC
0 points
in reply to: Larks’s comment on: Justification Through Pragmatism
That’s the problem with language. I meant: ∃x such that Px, in the first place, for which the negation is indeed: ∀x ¬Px

Maybe the sentence is ambiguous. I don’t mean “given anything, that one has the most basic ability to understand it”, but “that there is anything that one has the most basic ability to understand”. I would naturally read what I wrote the second way, but I included the negation specifically to make it clear that’s what I meant in the first place.

Irgy 17 Nov 2011 21:57 UTC
−1 points
in reply to: AmagicalFishy’s comment on: Justification Through Pragmatism
I don’t think I’m confused about this at all, I’m simply using the words in a much more specific way than you imagine. Yes “better” and “worse” are subjective and relative, but I am using them specifically with respect to the subject of oneself, and I’m using it specifically to compare entire future possible states of the world to other future possible states of the world. I mean “better” and “worse’ in the sense often described as “preferable”, with a sense of correctness to that preference as opposed to whim.

The question about God shows you’ve completely missed the point—which may be my fault as much as yours but there it is. To answer the question then, which may well help: It is not true to say that it’s impossible to make any progress or meaningful decisions without belief in God. The negation of “belief in God” does not prevent all possible progress, where as the negation of each of the examples I prevent does. Pascal’s Wager is also a form of pragmatism but is otherwise unrelated.

Irgy 20 Jan 2012 0:47 UTC
0 points
on: The Savage theorem and the Ellsberg paradox
It seems just a classic case of people being naturally unwilling to fully dive into the abstraction.

Firstly, the fully abstracted problem provides only indifference between the choices. So, as much as we are able to dive in and cut out all external influences in our thinking, even when we do, the problem at best tells us that we should be indifferent. So we’re indifferent, then we’re asked us to make a choice. If you’re the sort of person who, when faced with an indifferent choice will say “I am provably indifferent, and thus refuse to choose”, you will generally be less functional in life than somebody who just picks one. So we all make a choice, and I would say when it’s indifferent then any choice is as good as any other. My point is, since the abstract problem gives us nothing, then even an infinitessimal hole in our ability to accept the rules of the problem makes all the difference.

If you don’t accept the abstraction fully, then there’s plenty of reasons to make the choices people make, as other comments already mention—assuming the game is more likely rigged against you than for you, and risk aversion in the case where there at least might be multiple plays.

Of course, I do think there still is a thinking failure people are making here. People see there being two levels and don’t realise they can’t really be separated. The problem people think they are solving is something like the following: “I will give you $X, where is the number of balls of the colour(s) you choose in the urn”. Normally that would seem to be an equivalent problem, and in that problem risk aversion trumps indifference and people’s choices are, well, as rational as risk aversion anyway. Strangely, a decreasing average utility of larger amounts of money both justifies risk aversion while at the same time causes my money version of the problem to no longer be equivalent.

Overall, the only thing I think is ridiculous is the suggestion that rule 2 should be dropped because of this. The justification for rule 2 is not in any way damaged by this paradox. Rationality should not be defined as what reasonable people choose to do anyway, as people have been wrong before and will be wrong again. At best it shows that perfectly reasonable people can be infinitessimally irrational in a contrived corner case, at worst it shows nothing.

A more interesting setup would be with either 29 or 31 balls. In that case there’s a finite cost to the wrong decision. How many “reasonable” people still stick to their suboptimal choice in that case though?

Irgy 12 Mar 2012 1:10 UTC
3 points
on: Causal diagrams and software engineering
While your suggestion at the end fixes the theoretical, statistical problem, in practice I don’t think it would actually resolve the specific example question. The trouble is, you don’t know what’s wrong with the requirements until you’re well into the project, so asking at the start about requirements quality would not help. An alternative approach in this case would be to balance the biases by asking the people who gave the requirements what caused the project to fail. You would probably get a different answer, validating the point you’re making if so and providing further insight into the real problems.

Although what you suggest might at least qualitatively answer a probably more useful question: “In how many cases could examining the requirements more carefully have identified significant issues?”. Just as there’s no point using requirements as a scapegoat, there’s also no point being aware that they’re a real potential problem unless you can actually identify and fix them in time. Really what people need are the top “avoidable sources of project failure”, which is not the same as the top set of things to blame in hindsight even when it is valid to do so.

Irgy 13 Mar 2012 1:17 UTC
0 points
on: Decision Theories: A Less Wrong Primer
Personally I don’t see the examples given as flaws in Causal Decision Theory at all. The flaw is in the problem statements not CDT.

In the alien predictor example, the key question is “when does the agent set its strategy?”. If the agent’s strategy is set before the prediction is made, then CDT works fine. The agent decides in advance to commit itself to opening one box, the alien realises that, the agent follows through with it and gets $1,000,000. Which is exactly how humans win that game as well. If on the other hand the agent’s strategy is not set until after the prediction, well I ask you what is the alien actually predicting? The alien cannot prediction the agent’s choice, because we’ve just said the agent’s strategy is not defined yet. However, what the alien can predict is the process by which the agent’s strategy will be set. In that case, there is a meta-agent, which has a strategy of “Force the agent to use CDT” (or something like that). In that case, the alien is really predicting the meta-agent, and the meta-agent has made a bad decision. The reality in that case is that the meta-agent is the one playing the game. There’s no $1,000,000 in the box not because of anything the agent did, but because the meta-agent made a poor decision to commit to creating an agent that would open both boxes. By the time the agent starts to operate there’s already no money in the box and it’s actually doing the correct thing to at least get $1,000 anyway.

The confusion comes from the problem being framed in terms of the subgame of “one box or two”, when the real game being played is “pick a box-opening strategy for the alien to predict”, and CDT can solve that one perfectly well.

A similar problem occurs in the prisoner’s dilemna. Again the problem is “when is the agent’s strategy set?”, or in this case in particular “when is the opponent’s strategy set?”. If the opponent’s strategy is already set before the agent makes its decision, then defecting is correct. However, in the example you give the agent supposedly knows whether it’s playing against itself. The implication is that by changing its strategy it implicitly changes its opponent’s strategy as well. If the agent does not know this, then it has simply been misled. If it does know this, then CDT is perfectly capable of telling it to co-operate with itself to acheive the best payout. Again, the metagame is “pick a strategy that optimises the prisoner’s dilemna where you might be playing against yourself, or against other agents who are aware of your strategy etc.”. Once again CDT is perfectly capable of handling this metagame.

I’ve got nothing against the idea of decision theories that are robust to misleading problem statements, or that are “extended” to include awareness of the fact that they’re in a metagame, but the examples given here don’t demonstrate any flaws in CDT to me as such. They seem akin to a chess-playing agent that isn’t told the opponent has a queen, they’re simply getting the wrong answers because they’re solving the wrong problems.

I might be missing the point about the differences between these things, but since my understanding of the terminology (I was familiar with the concepts already) is based on this article it’s still a problem with the article. Or my reading comprehension skills I suppose, but I’ll stick with blaming the article for now.

Irgy 13 Mar 2012 20:30 UTC
0 points
in reply to: Vladimir_Nesov’s comment on: Decision Theories: A Less Wrong Primer
I effectively already go on to address that question in the rest of the paragraph. I see no meaningful difference between “strategy not set” and “agent not created”, if there is a difference to you please elborate.

At risk of repeating myself, to answer your question anyway, I ask: How can the alien successfully predict the strategy of something which has yet to even be created? If the alien is just guessing then opening both boxes is clearly correct anyway. Otherwise, the alien must know something about the process by which the agent is created. In this case, as I explain in the original comment, there is a meta-agent which is whatever creates the agent, and that is also what the alien is predicting the behaviour of. If there’s no money in the box it’s due to a poor meta-agent strategy which the agent then has no means to rectify. CDT seems to me perfectly capable of generating the correct strategies for both the agent and the meta-agent in this case.

Irgy 13 Mar 2012 21:45 UTC
21 points
on: Fallacies as weak Bayesian evidence
I don’t agree with your analysis of circular arguments. It seems to be saying electrons are more likely than God because we have a higher prior for electrons than for God. I don’t even think this is true, certainly before the discovery of both electrons and the bible not many people would have put a higher prior on tiny points of negative charge than a divine creator.

The real difference is not how well the evidence demonstrates the theories (as all that can really be said is that it’s “consistent” with them), but what the alternatives are and how well the evidence disproves them. In the case of God, the bible could well exist in its current form, as evidenced by the existence other religious books inconsistent with it (at least most of which must therefore have been created by man). On the other hand, without electrons there simply is no other explanation for the tracks. In this case the experiment knocks out all simpler alternatives. That’s the important difference—it’s in the alternatives not the theory.

Irgy 14 Mar 2012 3:33 UTC
3 points
in reply to: Vladimir_Nesov’s comment on: Decision Theories: A Less Wrong Primer
Ok, so firstly I now at least understand the difference you see between strategy-not-set and agent-not-created—that in only one case was there the clear potential for pre-commitment. I still think it’s beside the point, but that does require some explaining.

When I talk about a meta-agent, I don’t mean to imply the existence of any sort of intelligence or sentience for it, I simply mean there exists some process outside of the agent. The agent cannot gain or lose the $1,000,000 without changing that process, something which it has no control over. Whether this outside process is by way of an intelligent meta-agent that should have known better, the blind whims of chance, or the harshness of a deterministic reality is beside the point. Whether agents which choose one box do better than agents which choose both is a different question from whether it is correct to choose both boxes or not. When you switch a winning agent for a losing one, you simultaneously switch the situation that agent is presented with from a winning situation to a losing one.

It makes me think of the following paradox: Imagine that at some point in your life, God (or an alien or whatever) looks at whether you have been a perfect rationalist, and if so punches you in the nose. And assume of course that you are well aware of this. “Easy!” you think, “Just make one minor irrational decision and your nose will be fine”. But, that would of course be the perfectly rational thing to do, and so you still get punched in the nose. You get the idea. Should I now go on to say that rationalism doesn’t always win, and we therefore need to construct a new approach which does? (hint: of course not, but try and see the parallels here.)

In any case, rather than continue to argue over what is a fairly controvertial paradox even for humans let alone decision theories, let me take another tack here. If you are a firm believer in the predictive power of the alien then the problem is entirely equivalent to:
- Choice A: Get $1,000
- Choice B: Get $1,000,000
If CDT is presented with this problem, it would surely choose $1,000,000. The only way I see it wouldn’t is if CDT is defined as something like “Make the correct decision, except for being deliberately obtuse in insisting that causality is strictly temporal”, and then lo and behold it loses in paradoxes relating to non-temporal causality. If that’s what CDT effectively means, then fine, it loses. But to me, we don’t need a substantially different decision theory to resolve this paradox, we need to apply substantially the same decision theory to a different problem. To me, answering the question “what are the outcomes of my decisions” is part of defining the problem, not part of decision theory.

So the examples in the article still don’t motivate me to see a need for substantially new decision theories, just more clearly defined problems. If the other decision theories are all about saying “I’m solving the wrong problem” then that’s fine, and I can imagine potentially useful, but based on the examples given at least it still seems like the backwards way of going about things.

Irgy 17 Mar 2012 1:15 UTC
−2 points
in reply to: Vladimir_Nesov’s comment on: Decision Theories: A Less Wrong Primer
You can’t have it both ways. Either the agent’s behaviour is deterministic, or the alien cannot reliably predict it. If it is deterministic, what the source code is determines what the source code does, so it is contradictory to claim the agent can change one but not the other (if by “control” you mean “is responsible for” then that’s a different issue). If it is not deterministic, then aside from anything else the whole paradox falls apart.

Irgy 3 Apr 2012 3:22 UTC
1 point
on: Decision Theories: A Semi-Formal Analysis, Part I
Well, this article puts my confusion from the primer into context, and I think I understand the issue now.

The “problem” (and it’s ultimately not a problem, but I’ll get to that) I have is that the game these programs are playing is ill-posed, in that it’s not a closed system. It might appear as if they’re playing, for example, Prisoners’ Dilemna, but with access to each other’s source code they’re really playing some sort of Prisoners’ Meta-Dilemna, a fundamentally different game for all that it might seem superficially similar. Now I’m not enough of an expert to tell you exactly what defines a “well posed” game, but I’m quite confident this is not it. Or maybe I’m abusing the terminology a bit but hopefully my point is clear enough (if not I’ll elaborate). The distinction though is that the mechanism by which you make your decision on how to play should be outside the game itself, and therefore only observed indirectly.

One property this metagame has is that there is fundamentally no “correct” answer for what “decision theory” the programs should use. The proof is simple, imagine you have a correct answer D. Imagine then that this program faces a population of the following opponents (I’ll use the Prisoners’ Dilemna subgame as a simple example):
- Look at your opponent’s source code.
- If your opponent is using D, defect.
- Otherwise co-operate.
D is in fact the single worst strategy in this scenario by quite a margin, regardless of what it is.

But that’s fine, why would we even expect there to be a single correct strategy for every possible set of opponents? Of course there isn’t. This is where we take a step back to a game that is actually well posed and for which Naive Decision Theory works fine—the programmers’ game. Applying it there would go something like this:
- Create a probability distribution over the (potentially infinite) space of all possible programs that yours might be up against.
- Find the program which maximises the expected utility against that distribution of opponents.
For an infinite space of possible programs the second step might technically be impossible to perform optimally, but you do the best you can. There is of course a devil in the details of the first step in how you create that distribution, and when you look into it ultimately it’s a game between programmers, with its own Nash Equilibria and co-operative strategies and so on, not just a simple decision. But until you start having programmers read each others’ minds (in which case you’ve really just pointlessly shifted the whole thing up one meta-level) it’s at least a well-posed (if complex) game that the ordinary approaches should work for.

As I said earlier though the fact that I might call these games ill-posed doesn’t mean they’re not interesting. I’ll be fascinated to read about the strategies that people have devised for them and the attempts to be as general as possible. I’m a little concerned by the fact that the arguments so far seem to be along the lines of “You could do X, but then what if Y happens?”, purely because as I’ve shown above there is ultimately no end to that chain of logic. But maybe you’ll eventually step outside of it. My guess is that ultimately the “smarter” program wins. Certainly my generic counter-strategy has to generally be more complex (and therefore “smarter”? Maybe) than the “D” strategy it punishes in order to identify “D” in the first place.

In any case I look forward to finding out.

Irgy 3 Apr 2012 3:41 UTC
0 points
on: Fundamentals of kicking anthropic butt
Personally, I think saying there’s “no particular asymmetry” is dangerous to the point of being flat out wrong. The three possibilities don’t look the least bit symmetric to me, they’re all qualitatively quite different. There’s no “relevant”, asymmetry but how exactly do we know what’s relevant and what’s not? Applying symmetry in places it shouldn’t be applied is the key way in which people get these things wrong. The fact that it gives the right answer this time is no excuse.

So my challenge to you is, explain why the answer is ²⁄₃ without using the word “symmetry”.

Here’s my attempt: Start with a genuinely symmetric (prior) problem, then add the information. In this case, the genuinely symmetric problem is “It’s morning. What day is it and will/did the coin come up heads?”, while the information is “She just woke up, and the last thing she remembers is starting this particular bizzare coin/sleep game”. In the genuinely symmetric initial problem all days are equally likely and so are both coin flips. The process for applying this sort of additional information is to eliminate all scenarios that it’s inconsistent with, and renormalise what’s left. The information eliminates all possibilities except (Monday, heads), (Monday, tails), (Tuesday, tails) - and some more obscure possibilities of (for the sake of argument) negligable weight. These main three had equal weight before and are equally consistent with the new information so they have equal weight now.

Ok, I did use the word symmetry in there but only describing a different problem where it was safe. It’s still not the best construction because my initial problem isn’t all that well framed, but you get the idea.

Note that more generally you should ask for p(new information | scenario) and apply Bayes Rule, but anthropic-style information is a special case where the value of this is always either 0 or 1. Either it’s completely inconsistent with the scenario or guaranteed by it. That’s what leads to the simpler process I describe above of eliminating the impossible and simply renormalising what remains.

The good thing about doing it this way is that you can also get the exact answer for the case where she knows the coin is biased to land heads 52% of the time, where any idea that the scenario is symmetric is out the window.

Irgy 30 Jul 2012 22:56 UTC
1 point
on: Solve Psy-Kosh’s non-anthropic problem
The devil is in the assumption that everyone else will do the same thing as you for presumably the same reasons. “Nay” is basically a better strategy in general, though it’s not always right. The 90% odds of tails are correctly calculated.
- If you are highly confident that everyone else will say “yea”, it is indeed correct to say “yea”
- If you’re highly confident that everyone will say the same thing but you’re not sure what it is (an unlikely but interesting case), then make a guess (if it’s 50-50 then “yea” is better—the cutoff in fact is just a bit above 45% chance of all-yea over all-nay)
- If you don’t really know what the others will say, and they’re all acting independently, then odds are they’ve blown it by giving different answers whatever you do. In that case, 1-decider case dominates your decision, for essentially different reasons (so “nay” is better).
- The most difficult case is the more “natural” assumption that the other people are basically idential copies of you. In this case, what you have is a stochastic version of Newcomb’s Paradox. Actually it’s more like Newcomb’s Mugging. Once you know you are a decider, saying “yea” will genuinely give you a better average payout. But, when you’re not a decider, you’ll see more money donated if you are the sort of person who would say “nay” when you are a decider. It’s obvious why that’s true if you make the “identical copies” assumption explicitly, because your identical copy will say “nay” and cause a larger donation. So, you expect to make more money saying “yea” once you know you’re a decider, but you make more money overall if you’re a “nay”-sayer. The trick is that most of the extra money is made in the cases where you’re not a decider.
So, nay is definately the better precommitment, and TDT would presumably tell you to say nay. But the argument that, as an individual, your expected returns are better saying “yea” once you know you are a decider is still perfectly valid.

Irgy

Jus­tifi­ca­tion Through Pragmatism

Justification Through Pragmatism