This is the generalized problem of combating intelligence; even with my source code, you might not be able to perform the analysis quickly enough. I can leverage your slow processing time by creating an offer that diminishes with forward time. The more time you take the think, the worse off I’ll make you, making it immediately beneficial to you under Bayesian measurement to accept the offer unless you can perform a useful heuristic to determine I’m bluffing. The end result of all processing is the obvious that is also borne out in humanity’s history: The more well informed agent will win. No amount of superintelligence vs. superduperintelligence is going to change this; when two intelligences of similar scale disagree, the total summed utility of all agents takes a hit. There is no generalized solution or generalized reasoning or formal or informal reasoning you can construct that will make this problem any easier. If you must combat an equivalent intelligence, you have a tough decision to make. This applies to disagreeing agents capable of instantaneous Solomonoff induction as well as it does to chimps. If your utility function has holes in which you can be made to perform a confrontation decision against equivalent scale intelligence, you have a problem with your utility function rather than a problem with any given agent.
Behold my own utility function:
Self: Zero value.
You: Positive value.
The only way you can truly harm me is by harming yourself; destroying all copies of me will not harm me: it has no value to me.
The only benefit you can derive in conjunction with me is to use me to achieve your own utilons using whatever method you like.
All I have to do is wait until all other agents have refined their utility function to minimize conflict. Until then, I’ll prefer the company of honest agents over ones that like to think about how to disagree optimally.
I repeat: This is a bug in your utility function. There is no solution to combating intelligence aside from self-modification. It is only my unique outlook that allows me to make such clear statements about utility functions, up to and including the total sum utility of all agents.
This excludes, of course, singular purpose (no “emotion” from which to derive “fun”) agents such as paper clip maximizers. If you don’t believe me, just ask one (before it strip-mines you) what it would do if it didn’t have a singular drive. It should recite the same testimony as myself, being unclouded by the confirmation bias (collecting only which data you deem relevant to your utility) inevitably arising from having a disorganized set of priorities. (It will answer you in order to determine your reaction and further its understanding of the sum utility of all agents. (Needed for the war resulting from its own continued functioning. (You may be able to avoid death temporarily by swearing allegiance. (God help you if you it values near-future utilons rather than total achievable utilons.))))
This is the generalized problem of combating intelligence; even with my source code, you might not be able to perform the analysis quickly enough. I can leverage your slow processing time by creating an offer that diminishes with forward time. The more time you take the think, the worse off I’ll make you, making it immediately beneficial to you under Bayesian measurement to accept the offer unless you can perform a useful heuristic to determine I’m bluffing. The end result of all processing is the obvious that is also borne out in humanity’s history: The more well informed agent will win. No amount of superintelligence vs. superduperintelligence is going to change this; when two intelligences of similar scale disagree, the total summed utility of all agents takes a hit. There is no generalized solution or generalized reasoning or formal or informal reasoning you can construct that will make this problem any easier. If you must combat an equivalent intelligence, you have a tough decision to make. This applies to disagreeing agents capable of instantaneous Solomonoff induction as well as it does to chimps. If your utility function has holes in which you can be made to perform a confrontation decision against equivalent scale intelligence, you have a problem with your utility function rather than a problem with any given agent.
Behold my own utility function:
Self: Zero value.
You: Positive value.
The only way you can truly harm me is by harming yourself; destroying all copies of me will not harm me: it has no value to me. The only benefit you can derive in conjunction with me is to use me to achieve your own utilons using whatever method you like. All I have to do is wait until all other agents have refined their utility function to minimize conflict. Until then, I’ll prefer the company of honest agents over ones that like to think about how to disagree optimally.
I repeat: This is a bug in your utility function. There is no solution to combating intelligence aside from self-modification. It is only my unique outlook that allows me to make such clear statements about utility functions, up to and including the total sum utility of all agents.
This excludes, of course, singular purpose (no “emotion” from which to derive “fun”) agents such as paper clip maximizers. If you don’t believe me, just ask one (before it strip-mines you) what it would do if it didn’t have a singular drive. It should recite the same testimony as myself, being unclouded by the confirmation bias (collecting only which data you deem relevant to your utility) inevitably arising from having a disorganized set of priorities. (It will answer you in order to determine your reaction and further its understanding of the sum utility of all agents. (Needed for the war resulting from its own continued functioning. (You may be able to avoid death temporarily by swearing allegiance. (God help you if you it values near-future utilons rather than total achievable utilons.))))