When talking about upper bounds we cannot afford to just cheat saying humans can probably figure it out. That isn’t an upper bound—it is an optimistic prediction about human potential. Moreover we still need a definition of Friendliness built in so we can tell whether this thing that the human researchers come up with is Friendliness or some other thing with that name. (Even an extremely ‘meta’ reference to a definition is fine but still requires more bits to point to which part of the humans is able to define Friendliness.)
Upper bounds are hard. But yes, I know your understanding of the area is solid and your ancestor comment serves as the definitive answer to the question of the post. I disagree only with the statement as made.
When talking about upper bounds we cannot afford to just cheat saying humans can probably figure it out. That isn’t an upper bound—it is an optimistic prediction about human potential. Moreover we still need a definition of Friendliness built in so we can tell whether this thing that the human researchers come up with is Friendliness or some other thing with that name. (Even an extremely ‘meta’ reference to a definition is fine but still requires more bits to point to which part of the humans is able to define Friendliness.)
Upper bounds are hard. But yes, I know your understanding of the area is solid and your ancestor comment serves as the definitive answer to the question of the post. I disagree only with the statement as made.