Just to comment on the last example: I totally agree with your assessment of this.
In particular anything that involves Löb’s theorem or considerations about how an agent should reason when considering an identical copy of themselves is almost certainly impractical mathematical cloud-castle building. I don’t have anything against that type of activity as a pursuit in itself and engage in it quite a lot, but don’t have any illusions that it will solve any real problems in my lifetime.
Any actual AI will have extremely bounded rationality by those standards. Quite a few of the decision processes discussed in those articles are literally uncomputable, let alone able to be implemented in any hardware that can exist in the known universe. However, considering the much more relevant but thornier problems of resource-constrained decision making is not nearly so elegant and fun.
Just to comment on the last example: I totally agree with your assessment of this.
In particular anything that involves Löb’s theorem or considerations about how an agent should reason when considering an identical copy of themselves is almost certainly impractical mathematical cloud-castle building. I don’t have anything against that type of activity as a pursuit in itself and engage in it quite a lot, but don’t have any illusions that it will solve any real problems in my lifetime.
Any actual AI will have extremely bounded rationality by those standards. Quite a few of the decision processes discussed in those articles are literally uncomputable, let alone able to be implemented in any hardware that can exist in the known universe. However, considering the much more relevant but thornier problems of resource-constrained decision making is not nearly so elegant and fun.