Apologies for the delay: this is a long and thoughtful response I sadly don’t have the capacity to do justice to right now. Your iff seems to be central to your argument, but I am not convinced by either component. Firstly, for human + AI < AI to hold, we need AI to be less competent than the most competent humans, in any respect. It’s not obvious that AI alignment is at the right level of difficulty! Secondly, for human + AI + mindreading > AI to hold, it seems that AI must add a great deal of productivity. Why should I expect my subconscious thoughts and reflexes to be more like a 300% boost (for the typical employable researcher) than a 30% boost?
Apologies for the delay: this is a long and thoughtful response I sadly don’t have the capacity to do justice to right now. Your iff seems to be central to your argument, but I am not convinced by either component. Firstly, for human + AI < AI to hold, we need AI to be less competent than the most competent humans, in any respect. It’s not obvious that AI alignment is at the right level of difficulty! Secondly, for human + AI + mindreading > AI to hold, it seems that AI must add a great deal of productivity. Why should I expect my subconscious thoughts and reflexes to be more like a 300% boost (for the typical employable researcher) than a 30% boost?