Dangerous implies a threat. Conflicting goals aren’t sufficient to establish a threat substantial enough to need fighting or even shunning; that additionally requires the power to carry those goals to dangerous places.
Clippy’s not dangerous in that sense. It’d happily turn my mass into paperclips given the means and absent countervailing influences, but a non-foomed clippy with a basic understanding of human society meets neither criterion. With that in mind, and as it doesn’t appear to have the resources needed to foom (or to establish some kind of sub-foom paperclip regime) on its own initiative, our caution need only extend to denying it those resources. I even suspect I might be capable of liking it, provided some willing suspension of disbelief.
As best I can tell this isn’t like dealing with a psychopath, a person with human social aggressions but without the ability to form empathetic models or to make long-term game-theoretic decisions and commitments based on them. It’s more like dealing with an extreme ideologue: you don’t want to hand such a person any substantial power over your future, but you don’t often need to fight them, and tit-for-tat bargaining can be quite safe if you understand their motivations.
Ah. Generally I read “Clippy” as referring to User:Clippy or something like it, who’s usually portrayed as having human-parity intelligence and human-parity or fewer resources; I don’t think I’ve ever seen the word used unqualified to describe the monster raving superhuman paperclip maximizer of the original thought experiment.
...and here I find myself choosing my words carefully in order to avoid offending a fictional AI with a cognitive architecture revolving around stationary fasteners. Strange days indeed.
Dangerous implies a threat. Conflicting goals aren’t sufficient to establish a threat substantial enough to need fighting or even shunning; that additionally requires the power to carry those goals to dangerous places.
Clippy’s not dangerous in that sense. It’d happily turn my mass into paperclips given the means and absent countervailing influences, but a non-foomed clippy with a basic understanding of human society meets neither criterion. With that in mind, and as it doesn’t appear to have the resources needed to foom (or to establish some kind of sub-foom paperclip regime) on its own initiative, our caution need only extend to denying it those resources. I even suspect I might be capable of liking it, provided some willing suspension of disbelief.
As best I can tell this isn’t like dealing with a psychopath, a person with human social aggressions but without the ability to form empathetic models or to make long-term game-theoretic decisions and commitments based on them. It’s more like dealing with an extreme ideologue: you don’t want to hand such a person any substantial power over your future, but you don’t often need to fight them, and tit-for-tat bargaining can be quite safe if you understand their motivations.
I thought we were talking about a foomed/fooming Clippy.
Ah. Generally I read “Clippy” as referring to User:Clippy or something like it, who’s usually portrayed as having human-parity intelligence and human-parity or fewer resources; I don’t think I’ve ever seen the word used unqualified to describe the monster raving superhuman paperclip maximizer of the original thought experiment.
...and here I find myself choosing my words carefully in order to avoid offending a fictional AI with a cognitive architecture revolving around stationary fasteners. Strange days indeed.