Many errors committed by artificial intelligence stem not from the limitations of its intelligence, but from its inability to stop. While humans ask questions based on shared experiences when context is lacking during a conversation, AI attempts to fill those gaps with self-generated information.
This “over-inference” occurs because the system fails to distinguish between sufficient and insufficient information. Consequently, it draws confident conclusions despite a lack of key information, leading to malfunctions that deviate from the user’s intent.
The core of the problem is not the model’s performance, but the absence of a judgment mechanism. True safety does not begin with superior guessing ability, but with asking oneself the following questions before execution.
“Is the command just received perfect for execution?”
“Is there any missing essential information?”
“Were there any arbitrary judgments?”
To achieve this, a process is required where AI uses a self-generated checklist to verify the completeness of specifications before inference, and verifies them together with humans if any deficiencies are found. I believe that this simple process of ‘pausing’ and ‘checking’ is the key to building a system that is more accurate and secure than any sophisticated guesswork.
I think creating such a checklist would be a good idea.
-- It appears that the previous post was altered from its original intent during the translation process. I will rewrite it as concisely as possible.
From Inference to Verification
Many errors committed by artificial intelligence stem not from the limitations of its intelligence, but from its inability to stop. While humans ask questions based on shared experiences when context is lacking during a conversation, AI attempts to fill those gaps with self-generated information.
This “over-inference” occurs because the system fails to distinguish between sufficient and insufficient information. Consequently, it draws confident conclusions despite a lack of key information, leading to malfunctions that deviate from the user’s intent.
The core of the problem is not the model’s performance, but the absence of a judgment mechanism. True safety does not begin with superior guessing ability, but with asking oneself the following questions before execution.
“Is the command just received perfect for execution?”
“Is there any missing essential information?”
“Were there any arbitrary judgments?”
To achieve this, a process is required where AI uses a self-generated checklist to verify the completeness of specifications before inference, and verifies them together with humans if any deficiencies are found. I believe that this simple process of ‘pausing’ and ‘checking’ is the key to building a system that is more accurate and secure than any sophisticated guesswork.
I think creating such a checklist would be a good idea.
-- It appears that the previous post was altered from its original intent during the translation process. I will rewrite it as concisely as possible.