Really? Where? I just reread it with that in mind and I still couldn’t find it. The closest I came was that he once used the term “sentient simulation”, which is at least technically broad enough to cover both. He does make a point there about sentience being something which may not exactly match our concept of a human, is that what you’re referring to? He then goes on to talk about this concept (or, specifically, the method needed to avoid it) as a “nonperson predicate”, again suggesting that what’s important is whether it’s like a human-like rather than anything more fundamental. I don’t see how you could think “nonperson predicate” is covering both human and nonhuman intelligence equally.
“Person” seems to be used here as the philosophical term meaning something like “sentient entity with moral value”. Personhood is not limited to human beings.
ETA: Also, wrt the AI itself, the directly next two articles in this sequence explicitly deal with the issue of making the AI itself nonsentient, as I’m surprised to find a comment from myself in 2011 pointing out. Did you really not read the surrounding articles?
Is a human mind the simplest possible mind that can be sentient? What if, in the course of trying to model its own programmers, a relatively younger AI manages to create a sentient simulation trapped within itself? How soon do you have to start worrying? Ask yourself that fundamental question, “What do I think I know, and how do I think I know it?”
I read this as being simpler than a real human mind. Since it’s simpler, the abstractions used are going to be imperfect, and the design would end up being something that is in some way artificial. It’s not as explicit as I said, but I still think the implication is pretty strong.
I’ve actually lost track of how this impacts my original point. As stated, it was that we’re worrying about the ethical treatment of simulations within an AI before worrying about the ethical treatment of the simulating AI itself. Whether the simulations considered include AIs as well as humans is an entirely orthogonal issue.
I went on in other comments to rant a bit about the human-centrism issue, which your original comment seems more relevant to though. I think you’ve convinced me that the original article was a little more open to the idea of substantially nonhuman intelligence than I might have initially credited it, but I still see the human-centrism as a strong theme.
My point is he’s clearly not drawing a box tightly around what’s human or not. If he’s concerned with clearly-sub-human AI, then he’s casting a significantly wider net than it seems you’re assuming he is. And considering that he’s written extensively on the variety of mind-space, assuming he’s taking a tightly parochial view is poorly founded.
What? That’s completely irrelevant to the question at hand.
By considering the question of whether simpler-than-human minds are possible in this context, it’s clear that Eliezer was thinking about the question and giving them moral weight. He doesn’t need to ANSWER the question I was posing to make that much clear.
Really? Where? I just reread it with that in mind and I still couldn’t find it. The closest I came was that he once used the term “sentient simulation”, which is at least technically broad enough to cover both. He does make a point there about sentience being something which may not exactly match our concept of a human, is that what you’re referring to? He then goes on to talk about this concept (or, specifically, the method needed to avoid it) as a “nonperson predicate”, again suggesting that what’s important is whether it’s like a human-like rather than anything more fundamental. I don’t see how you could think “nonperson predicate” is covering both human and nonhuman intelligence equally.
“Person” seems to be used here as the philosophical term meaning something like “sentient entity with moral value”. Personhood is not limited to human beings.
ETA: Also, wrt the AI itself, the directly next two articles in this sequence explicitly deal with the issue of making the AI itself nonsentient, as I’m surprised to find a comment from myself in 2011 pointing out. Did you really not read the surrounding articles?
I read this as being simpler than a real human mind. Since it’s simpler, the abstractions used are going to be imperfect, and the design would end up being something that is in some way artificial. It’s not as explicit as I said, but I still think the implication is pretty strong.
I’ve actually lost track of how this impacts my original point. As stated, it was that we’re worrying about the ethical treatment of simulations within an AI before worrying about the ethical treatment of the simulating AI itself. Whether the simulations considered include AIs as well as humans is an entirely orthogonal issue.
I went on in other comments to rant a bit about the human-centrism issue, which your original comment seems more relevant to though. I think you’ve convinced me that the original article was a little more open to the idea of substantially nonhuman intelligence than I might have initially credited it, but I still see the human-centrism as a strong theme.
My point is he’s clearly not drawing a box tightly around what’s human or not. If he’s concerned with clearly-sub-human AI, then he’s casting a significantly wider net than it seems you’re assuming he is. And considering that he’s written extensively on the variety of mind-space, assuming he’s taking a tightly parochial view is poorly founded.
“Is a human mind the simplest possible mind?”
“But if it was simpler, it wouldn’t be human!”
Downvoted.
What? That’s completely irrelevant to the question at hand.
By considering the question of whether simpler-than-human minds are possible in this context, it’s clear that Eliezer was thinking about the question and giving them moral weight. He doesn’t need to ANSWER the question I was posing to make that much clear.
Wait, what?
*Clicks “Show more comments above.”
Oops. I thought you were replying to the quoted text. Upvoted and retracted my comment.