Could I add a brief thought? Even if you could program an AI with no bugs, that AI could make “mistakes”. What we consider a mistake depends on the practical purpose we have in mind for the AI. What we consider a bug depends on the operational purpose of the segment of code the bug appears in. But the operational purpose may not match up with our practical purpose. So code which achieves its operational purpose might not achieve its practical purpose.
Let me take this a bit deeper. The programmer may have an overarching intended purpose for the entire project of building the AI. They may also have an intended purpose for the entirety of the code written. They may also have an intended purpose for the programs A, B, C… in the code. They may… and so forth. At every stage of increasing generality, there is a potential for the more general purpose to not be fulfilled. Your programs might do what you want them to, but they might fail in combination to detect an absolute denial macro. The entirety of your code might resemble an AI, but it would take a computer far more powerful than any in existence to run on. You might get the whole project up and working, but only for the AI to decide to commit suicide. Or, more relevantly, you might get the whole project working, but the AI turns out to be dumb, because the way you thought an AI ought to think in order to be clever didn’t work out. Your intended purpose to create an AI which thought in the way you intended was successful, but one of the more general purposes, to create an AI that was clever, failed.
So it would seem that bug checking is more prone to human error than you implied, especially as intended purposes are themselves often vague. I don’t claim, however, that these challenges are insurmountable. Also, if anyone is uncomfortable with the phrase “intended purpose” I used, feel free to replace with “what the programmer had in mind”, as that is all I meant by it.
Hi. Checking back on this account on a whim after a long time of not using it. You’re right. 2012!Mestroyer was a noob and I am still cleaning up his bad software.
Could I add a brief thought? Even if you could program an AI with no bugs, that AI could make “mistakes”. What we consider a mistake depends on the practical purpose we have in mind for the AI. What we consider a bug depends on the operational purpose of the segment of code the bug appears in. But the operational purpose may not match up with our practical purpose. So code which achieves its operational purpose might not achieve its practical purpose.
Let me take this a bit deeper. The programmer may have an overarching intended purpose for the entire project of building the AI. They may also have an intended purpose for the entirety of the code written. They may also have an intended purpose for the programs A, B, C… in the code. They may… and so forth. At every stage of increasing generality, there is a potential for the more general purpose to not be fulfilled. Your programs might do what you want them to, but they might fail in combination to detect an absolute denial macro. The entirety of your code might resemble an AI, but it would take a computer far more powerful than any in existence to run on. You might get the whole project up and working, but only for the AI to decide to commit suicide. Or, more relevantly, you might get the whole project working, but the AI turns out to be dumb, because the way you thought an AI ought to think in order to be clever didn’t work out. Your intended purpose to create an AI which thought in the way you intended was successful, but one of the more general purposes, to create an AI that was clever, failed.
So it would seem that bug checking is more prone to human error than you implied, especially as intended purposes are themselves often vague. I don’t claim, however, that these challenges are insurmountable. Also, if anyone is uncomfortable with the phrase “intended purpose” I used, feel free to replace with “what the programmer had in mind”, as that is all I meant by it.
Hi. Checking back on this account on a whim after a long time of not using it. You’re right. 2012!Mestroyer was a noob and I am still cleaning up his bad software.