Paul_Crowley2

Karma: 129

Paul_Crowley2 27 Dec 2008 12:43 UTC
0 points
on: Nonsentient Optimizers
But we want them to be sentient. These things are going to be our cultural successors. We want to be able to enjoy their company. We don’t want to pass the torch on to something that isn’t sentient. If we were to build a nonsentient one, assuming such a thing is even possible, one of the first things it would do would be start working on its sentient successor.

In any case, it seems weird to try and imagine such a thing. We are sentient entirely as a result of being powerful optimisers. We would not want to build an AI we couldn’t talk to, and if it can talk to us as we can talk to each other, it’s hard to see what aspect of sentience it could be lacking. At first blush it reads as if you plan to build an AI that’s just like us except it doesn’t have a Cartesian Theatre.

Paul_Crowley2 27 Dec 2008 12:34 UTC
0 points
on: Nonperson Predicates
“by the time the AI is smart enough to do that, it will be smart enough not to”

I still don’t quite grasp why this isn’t an adequate answer. If an FAI shares our CEV, it won’t want to simulate zillions of conscious people in order to put them through great torture, and it will figure out how to avoid it. Is it simply that it may take the simulated torture of zillions for the FAI to figure this out? I don’t see any reason to think that we will find this problem very much easier to solve than a massively powerful AI.

I’m also not wholly convinced that the only ethical way to treat simulacra is never to create them, but I need to think about that one further.

Paul_Crowley2 24 Dec 2008 2:17 UTC
1 point
on: The Fear of Common Knowledge
Posting much too late, but about the Emperor’s New Clothes: I had always interpreted the story to mean not that people stopped believing in the clothes when the little boy spoke out, but that it hadn’t quite crossed people’s minds until that moment that whether or not the King was clothed, he appeared naked to that boy, and that wasn’t how things should be. Everyone laughs because they think, I too can see the nakedness of the King; only then do they realise that their neighbour can also see it, and only after that do they see that there are no clothes to see.

Paul_Crowley2 24 Dec 2008 1:46 UTC
10 points
on: Imaginary Positions
Very recently experienced exactly this phenomenon: someone discussing atheists who think “all religion/religious stuff is bad” to the inclusion of, for example, the music of Bach, or drinking and celebrating at Christmas. They seemed convinced that such atheists exist, and I doubt it, or at least I have never heard of them or met them, and I know for a fact that for example all four horsemen of atheism have made explicit statements to the contrary.

Your disclaimer is an annoying one to have to make, and of course this problem comes up whenever this move is made in discussion; your counterpart says “well, but some singularitarians believe that, don’t they?” and you can’t actually prove there are none, and you have the sneaking fear that given the vastness of the Internet a judicious Google search might just turn up someone crazy enough to vindicate them; but of course a handful of anonymous loons on the Internet sought specifically for their particular brand of madness does not a position worthy of discussion make.

Paul_Crowley2 18 Dec 2008 23:03 UTC
2 points
on: Prolegomena to a Theory of Fun
I can’t believe the discussion has got this far and no-one has mentioned The Land of Infinite Fun.

Paul_Crowley2 17 Nov 2008 20:32 UTC
0 points
on: Whither OB?
There is room to do vastly better than what is usually used for community content finding, and it’s a great mystery to me how little explored this area is. If things have moved forward significantly since Raph Levien’s work on attack resistant trust metrics, I haven’t heard about it.

Good software to support rational discussion would be a huge contribution to thought.

Paul_Crowley2 9 Nov 2008 21:15 UTC
1 point
on: Ask OB: Leaving the Fold
There’s a whole world of atheist blogging and writing out there that might also be worth tapping into for advice from others who’ve been there. See this collection of deconversion stories for example.

That sounds like a really tough spot. I hope you find advice that can help.

Paul_Crowley2 31 Oct 2008 19:14 UTC
14 points
on: Mundane Magic
Possession of a single Eye is said to make the bearer equivalent to royalty.

I approve.

Paul_Crowley2 26 Oct 2008 19:44 UTC
0 points
on: Aiming at the Target
http://en.wikipedia.org/wiki/Intentional_stance

Paul_Crowley2 21 Oct 2008 19:49 UTC
4 points
on: Prices or Bindings?
Crossman: there’s a third argument, which is that even if the consequences of keeping the secret are overall worse than those of betraying the confidence even after the effect you discuss, turning yourself into someone who will never betray these secrets no matter what the consequences and advertising yourself as such in an impossible-to-fake way may overall have good consequences. In other words, you might turn away from consequentialism on consequentialist grounds.

Another example where unfakeably advertising irrationality can (at least in theory) serve you is threats. My only way of stopping you from taking over the world is that I have the power to destroy the world and you. Now, if you take over the world, there’s no possible advantage to destroying it, so I won’t, so you can take the world over. But if I put a lunatic in charge of the button who believably will carry out the threat, you will be deterred; the same applies if I can become that lunatic.

However, overall I think that the arguments against turning yourself into a lunatic are pretty strong, and in fact I suspect that consequentialism has the best consequences.

Paul_Crowley2 18 Oct 2008 16:27 UTC
13 points
on: Dark Side Epistemology
- There’s a huge conspiracy covering it up
- Well, that’s just what one of the Bad Guys would say, isn’t it?
- Why should I have to justify myself to you?
- Oh, you with your book-learning, you think you’re smarter than me?
- They said that to Einstein and Galileo!
- That’s a very interesting question, let me show you the entire library that’s been written about it (where if there were a satisfactory answer it would be shortish)
- How can you be so sure?

Paul_Crowley2 15 Sep 2008 8:51 UTC
18 points
on: My Childhood Death Spiral
Don’t think “silver spoons”, think “clean drinking water”.

Paul_Crowley2 15 Sep 2008 8:05 UTC
8 points
on: My Childhood Death Spiral
I like “we are the cards we are dealt”, which expresses nicely a problem with common ideas of blame and credit. I disagree that intelligence is the unfairest card of all - I think that a relatively dim person born into affluence in the USA has a much better time of it than a smart person born into poverty in the Congo.

Paul_Crowley2 4 Sep 2008 8:27 UTC
9 points
on: The True Prisoner’s Dilemma
Interesting. There’s a paradox involving a game in which players successively take a single coin from a large pile of coins. At any time a player may choose instead to take two coins, at which point the game ends and all further coins are lost. You can prove by induction that if both players are perfectly selfish, they will take two coins on their first move, no matter how large the pile is. People find this paradox impossible to swallow because they model perfect selfishness on the most selfish person they can imagine, not on a mathematically perfect selfishness machine. It’s nice to have an “intuition pump” that illustrates what genuine selfishness looks like.

Paul_Crowley2 10 Aug 2008 8:45 UTC
1 point
on: Sorting Pebbles Into Correct Heaps
Are you arguing that a few simple rules describe what we’re all trying to get at with our morality? That everyone’s moral preference function is the same deep down? That anything that appears to be a disagreement about what is desirable is actually just a disagreement about the consequences of these shared rules, and could therefore always be resolved in principle by a discussion between any two sufficiently wise, sufficiently patient debaters? And that moral progress consists of the moral zeigeist moving closer to what those rules capture?

That certainly would be convenient for the enterprise of building FAI.

Paul_Crowley2 16 Jul 2008 9:01 UTC
0 points
on: Whither Moral Progress?
Paul, do you think that your own morality is optimum or can you conceive of someone more moral than yourself—not just a being who better adheres to your current ideals, but a being with better ideals than you?

Yes I can.

If you take the view that ethics and aesthetics are one and the same, then in general it’s hard to imagine how any ideals other than your own could be better than your own for the obvious reason that I can only measure them against my own.

What interests me about the rule I propose (circular preferences are bad!) is that it is exclusively a meta-rule—it cannot measure my behavour, only my ideals. It provides a meta-ethic that can show flaws in my current ethical thinking, but not how to correct them—it provides no guidance on which arrow in the circle needs to be reversed. And I think it covers the way in which I’ve been persuaded of moral positions in the past (very hard to account for otherwise) and better yet allows me to imagine that I might be persuaded of moral points in the future, though obviously I can’t anticipate which ones.

If I can imagine that through this rule I could be persuaded to take a different moral stance in the future, and see that as good, then I’m definitely elevating a different set of ideals—my imagined future ideals—over my current ideals.

Paul_Crowley2 16 Jul 2008 7:53 UTC
2 points
on: Whither Moral Progress?
I’m by no means sure that the idea of moral progress can be salvaged. But it might be interesting to try and make a case that we have fewer circular preferences now than we used to.

Paul_Crowley2 11 May 2008 11:25 UTC
4 points
on: If Many-Worlds Had Come First
The “One Christers” are a nice SF touch.

Paul_Crowley2 27 Apr 2008 9:33 UTC
0 points
on: On Being Decoherent
It’s not known whether the Universe is finite or infinite, this article gives more details:

http://en.wikipedia.org/wiki/Shape_of_the_Universe

If the Universe is infinite, then it has always been so even from the moment after the Big Bang; an infinite space can still expand.

It hadn’t quite sunk in until this article that looked at from a sum-over-histories point of view, only identical configurations interfere; that makes decoherence much easier to understand.

Paul_Crowley2 3 Apr 2008 7:02 UTC
−1 points
on: Reductive Reference
Would this get easier or harder if you started with, say, gliders in Conway’s Life?