I think this essay drifts considerably further away from SIAI/LW thinking than his story does, though I might have forgotten things.
Actually, given a moment to reflect, I’m more confusing the essay’s points and my own impressions of the story. If he thought like this while writing the novel, then he spectacularly failed to reach me. For that I’m glad.
Ungrowth wasn’t talked about in the novels. I remember the opposite complaint: that the overly strict implementation of the Three Laws turned humanity into kittens, with the wireheads at the extreme. Ungrowth sounds almost as bad as Peer’s arbitrary obsessions in Permutation City.
Holding all other implementation details equal, Lawrence’s insistence that PI not look into people’s brains results in a much better world than not. I get the impression that Roger thinks his genie could have handled people better if it analyzed them that deeply.
The critique of the Three Laws as portrayed should have focused not on how limited it is, but on how restrictive it is. A premise Roger disagrees with when saying that Lawrence did not install a more robust ethical system when his design allowed for it. The star map above Lawrence’s house gave us a glimpse of the design that made it clear how he messed up his design not just in creating a thoughtless genie, but in not allowing corrections as the everything the AI believed was not only interdependent but centered on those three pillars.
Edit: There were more words here, but your later emphasis confuses me. I’m going to pretend you didn’t do that. If I’m not being clear here, please help me help me help you.
I think this essay drifts considerably further away from SIAI/LW thinking than his story does, though I might have forgotten things.
Actually, given a moment to reflect, I’m more confusing the essay’s points and my own impressions of the story. If he thought like this while writing the novel, then he spectacularly failed to reach me. For that I’m glad.
I think you’re speaking too abstractly to agree or disagree with your value judgement.
Ungrowth wasn’t talked about in the novels. I remember the opposite complaint: that the overly strict implementation of the Three Laws turned humanity into kittens, with the wireheads at the extreme. Ungrowth sounds almost as bad as Peer’s arbitrary obsessions in Permutation City.
Holding all other implementation details equal, Lawrence’s insistence that PI not look into people’s brains results in a much better world than not. I get the impression that Roger thinks his genie could have handled people better if it analyzed them that deeply.
The critique of the Three Laws as portrayed should have focused not on how limited it is, but on how restrictive it is. A premise Roger disagrees with when saying that Lawrence did not install a more robust ethical system when his design allowed for it. The star map above Lawrence’s house gave us a glimpse of the design that made it clear how he messed up his design not just in creating a thoughtless genie, but in not allowing corrections as the everything the AI believed was not only interdependent but centered on those three pillars.
Edit: There were more words here, but your later emphasis confuses me. I’m going to pretend you didn’t do that. If I’m not being clear here, please help me help me help you.