wasn’t it agreed-upon by FAI researchers that “Friendliness” as a supergoal
meant that the AI would find ways to do things that are “better” for humanity
overall in its prediction of the grand schemes of things.
This would include “being a good galactic citizen” with no specific preference
for humanity if the freedom, creativity, fairness, public perception by aliens, or
whatever other factor of influence led this goal to being superior in terms of
achieving human values and maximizing collective human utility.
I’m glad to hear it.
But I think there is a distinction here worth noting, between two positions:
POSITION ONE—Make “Be a good galactic citizen” be the supergoal if and only if setting that as the supergoal is the action that maximises the chances of the AI, in practice, ending up doing stuff to help humanity in the long term, once you take interfering aliens, etc into account
and
POSITION TWO—Make “Be a good galactic citizen” be the supergoal, even if that isn’t quite as certain an approach to helping humanity in particular, as setting “be friendly to humanity” as the supergoal would be.
Why on earth would anyone suggest that AI researchers follow an approach that isn’t the absolute safest for humanity? That’s a big question. But one I think worth considering, if we open the possibility that there is a bit of wiggle room for setting a supergoal that will still be ok for humanity, but be slightly more moral.
I’m glad to hear it.
But I think there is a distinction here worth noting, between two positions:
POSITION ONE—Make “Be a good galactic citizen” be the supergoal if and only if setting that as the supergoal is the action that maximises the chances of the AI, in practice, ending up doing stuff to help humanity in the long term, once you take interfering aliens, etc into account
and
POSITION TWO—Make “Be a good galactic citizen” be the supergoal, even if that isn’t quite as certain an approach to helping humanity in particular, as setting “be friendly to humanity” as the supergoal would be.
Why on earth would anyone suggest that AI researchers follow an approach that isn’t the absolute safest for humanity? That’s a big question. But one I think worth considering, if we open the possibility that there is a bit of wiggle room for setting a supergoal that will still be ok for humanity, but be slightly more moral.
You know: You don’t need to comment your post if you want to extend it. You may edit it. It seems customary to add it with a tag like
EDIT: …
--
Sorry. You are more senior than me. I confused you with a newbie. You will have your reason