language-models if they are AGI would surely surpass human-level understanding of language, humans need language for communication and book-keeping, “words” in any language are mostly for interesting abstractions from a human point-of-view, as for language models it need not have any language since it doesn’t have an internal dialogue like humans do.
As it reaches a certain level of intelligence it starts forming increasingly complex abstractions that don’t have (won’t have) any vocabulary. It would be impossible to interpret its reasoning, and the only way left is to accept it.
language-models if they are AGI would surely surpass human-level understanding of language, humans need language for communication and book-keeping, “words” in any language are mostly for interesting abstractions from a human point-of-view, as for language models it need not have any language since it doesn’t have an internal dialogue like humans do.
As it reaches a certain level of intelligence it starts forming increasingly complex abstractions that don’t have (won’t have) any vocabulary. It would be impossible to interpret its reasoning, and the only way left is to accept it.