Okay, so no one gets their driver’s license until they’ve built their own Friendly AI, without help or instruction manuals. Seems to me like a reasonable test of adolescence.
Does this assume that they would be protected from any consequences of messing the Friendliness up and building a UFAI by accident? I don’t see a good solution to this. If people are protected from being eaten by their creations, they can slog through the problem using a trial-and-error approach through however many iterations it takes. If they aren’t, this is going to be one deadly test.
Sebastian_Hagen2 comments on Shut up and do the impossible!