I would expect one’s ability to “correlate knowledge” to be strongly related to one’s intelligence, but it’s not clear to me that a super-LLM would have a near-zero probability of accidentally deleting humanity. What level of knowledge-correlation ability is required to consistently avoid fatal mistakes? The level of the average LW user is not high enough, because we still do things like poison ourselves.
I would expect one’s ability to “correlate knowledge” to be strongly related to one’s intelligence, but it’s not clear to me that a super-LLM would have a near-zero probability of accidentally deleting humanity. What level of knowledge-correlation ability is required to consistently avoid fatal mistakes? The level of the average LW user is not high enough, because we still do things like poison ourselves.