I’d love to see Yudkowsky try to convince ChatGPT why check_legality wouldn’t work!
Would there be anything specifically illegal about an AI paperclip maximizer executing a plan that would kill billions of people? It’s not legally a person, and therefore cannot be tried for committing a crime AFAIK (civil forfeiture edge cases aside). It’s not like we ever brought COVID or a hurricane to court, either.
Agree with this. Law is downstream of a particular medium-scale ontology of human agency. A paperclip maximizer, as mythologized, would be working with a different notion of agency, by definition.
Querying check_legality(“clippy’s plan”), would be like checking the temperature of the number “100″. Sure, it might kinda sound like its hot, or illegal, but that’s not the kind of input that check_legality() currently takes.
check_legality() can’t even handle the agency of nation states …
Would there be anything specifically illegal about an AI paperclip maximizer executing a plan that would kill billions of people? It’s not legally a person, and therefore cannot be tried for committing a crime AFAIK (civil forfeiture edge cases aside). It’s not like we ever brought COVID or a hurricane to court, either.
Agree with this. Law is downstream of a particular medium-scale ontology of human agency. A paperclip maximizer, as mythologized, would be working with a different notion of agency, by definition.
Querying check_legality(“clippy’s plan”), would be like checking the temperature of the number “100″. Sure, it might kinda sound like its hot, or illegal, but that’s not the kind of input that check_legality() currently takes.
check_legality() can’t even handle the agency of nation states …