In general terms proof finding has exponential complexity while verification is polinomial. “Difficulty” in checking formal proofs is not a thing. Issue is that large portion of science has much weaker reasoning than formal systems. There we have somewhat vague/intuitive proofs (compared to fomal math proofs) that relay on common sense priors. That is possible route for falsehoods from LMs, but again it is not an issue really. Humans have deliberetly abused the scientific process since its inception. As long as we relay on the process as it is (instead of taking AI or whatever authority for its word) we are fine.
Also singularity is not a thing one should fear. Most interesting problems have exp complexity and this stands as had limitation for any system. There is pretty much no way around it. At best LMs will be human like in terms of AGI. They can be X times faster, but that advantage vanishes in front of problems with exp complexity.
language-models if they are AGI would surely surpass human-level understanding of language, humans need language for communication and book-keeping, “words” in any language are mostly for interesting abstractions from a human point-of-view, as for language models it need not have any language since it doesn’t have an internal dialogue like humans do.
As it reaches a certain level of intelligence it starts forming increasingly complex abstractions that don’t have (won’t have) any vocabulary. It would be impossible to interpret its reasoning, and the only way left is to accept it.
In general terms proof finding has exponential complexity while verification is polinomial. “Difficulty” in checking formal proofs is not a thing. Issue is that large portion of science has much weaker reasoning than formal systems. There we have somewhat vague/intuitive proofs (compared to fomal math proofs) that relay on common sense priors. That is possible route for falsehoods from LMs, but again it is not an issue really. Humans have deliberetly abused the scientific process since its inception. As long as we relay on the process as it is (instead of taking AI or whatever authority for its word) we are fine.
Also singularity is not a thing one should fear. Most interesting problems have exp complexity and this stands as had limitation for any system. There is pretty much no way around it. At best LMs will be human like in terms of AGI. They can be X times faster, but that advantage vanishes in front of problems with exp complexity.
language-models if they are AGI would surely surpass human-level understanding of language, humans need language for communication and book-keeping, “words” in any language are mostly for interesting abstractions from a human point-of-view, as for language models it need not have any language since it doesn’t have an internal dialogue like humans do.
As it reaches a certain level of intelligence it starts forming increasingly complex abstractions that don’t have (won’t have) any vocabulary. It would be impossible to interpret its reasoning, and the only way left is to accept it.