It is the “learn to walk before you start learning to run and jump” approach.
Imagine that you are the first human who ever tried to invent math. You have invented the “2 + 2 = 4” theorem, and it seems to work! Two stones together with other two stones always make four stones. And two bananas together with other two bananas always make four bananas. Seems like you have found a powerful abstraction behind stones and bananas—who knows what else might be obeying the same rules. Excited you run towards your tribe members to share your discovery...
...but they dismiss it as worthless. “No two stones are same. A sharp stone is a more useful tool than a blunt stone, and your theory ignores that. A banana can be eaten. Actually, bananas are only useful for eating, and you completely ignore this essential part. And have you ever tried to ‘add’ two and two raindrops? Obviously, this ‘mathematics’ of yours has no future, and is just a waste of time” they say.
Well, the answer is that you need to apply the “2 + 2 = 4” only in the situations where it is appropriate, and not where it’s not. And telling the difference may be tricky. And it may be easy to underestimate some problem. But still, the equation is useful in some contexts, which makes it useful to know it.
Does refusing to know “2 + 2 = 4” because of these practical concerns make you stronger or weaker?
I don’t understand why: other agents trust that your source code is what you say it is
It’s an assumption to simplify the problem. First we learn to solve the “simple” problem, then we move to the more complex ones. If we can’t solve the simpler problem yet, would jumping to the more complex one make us progress faster?
Would a recommendation to skip solving the simplified problems, because they don’t include all the messy details, more likely lead to prople solving the hard problems faster, or giving up?
I think pragmatism make a better core as it makes less assumptions about the nature of reality, with other stuff layered on top.
How specifically are you planning to use this insight in constructing a Friendly superintelligent AI?
Ugh can use it right away for counting days. Source code based decision theory not so much. There aren’t the societies based on agents that can read each others source code, so I can’t try and predict them with source code based decision theories. It seems like it is mathematically interesting thing though, so it is still interesting. I just don’t want it to be a core part of our sole pathway to try and solve the AI problem.
Would a recommendation to skip solving the simplified problems, because they don’t include all the messy details, more likely lead to prople solving the hard problems faster, or giving up?
Perhaps it would lead people to avoid trying to find optimal decision theories and accept the answer that the best decision theory depends on the circumstances. Then we can figure out what our circumstances are and find good decisions theories for those. And create designs that can do similarly.
Like the best search algorithm is context dependent, where even algorithms of a worst complexity class can be better due to memory locality and small size.
How specifically are you planning to use this insight in constructing a Friendly superintelligent AI?
Supposition> If answers to how decisions are made (and a whole host of other problems) are contextual and complex then it is worth trading information about what answers they have found within their context.
Figure the control flows between parts of the human brain that means they manage to roughly align themselves into a coherent entity (looking at dopamine etc).
Apply the same informational constraints to computers so that we can link a computer up to a human (it doesn’t need to be physically as long as the control flows work). The computer should be aligned with the user as much as a part of the brain is aligned to another part (which is hopefully sufficient).
While this is in an embryonic stage get as many people from all around the world to use it .
Hard take off (I think this less likely due to the context sensitivity things I have described, but still possible)- it is worthwhile to trade between agents so groups of human/computers that co-operate and trade information advance quicker. than loan rogue agents. If we have done the alignment work right then it is likely the weight of them will be able to squash any human inimical systems that appear.
This pathway makes no sense from people that expect there to be winner take all optimal AI designs as any one embryonic system might find the keys to the future and take it over. But if that is not the way the world works....
There aren’t the societies based on agents that can read each others source code,
Human beings can probabilistically read each others’ source code. That’s why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.
Human beings can probabilistically read each others’ source code
This seems like a weird way of say, humans can make/refine hypotheses about other agents. What does talking about source code give you?
That’s why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.
Tit for tat (which seems like revenge) works in normal game theories for IPD (of infinite length) which is a closer to what we experience in everyday life. I thought Non-causal decision theories are needed for winning on one-shots?
In the case of humans, “talking about source code” is perhaps not that useful, though we do have source code, it’s written in quaternary and has a rather complex probabilistic compiler. And that source code was optimized by a purely causal process, demonstrating the fact that causal decision theory agents self modify into acausal decision theory agents in many circumstances.
Revenge and anger work for one shot problems, for example if some stranger comes and sexually assaults your wife, they cannot escape your wrath by “saying oh it’s only one shot, I’m a stranger in a huge city you’ll never see me again so there’s no point taking revenge”. You want to punch the in the face as an end in itself now, this is a simple way of our brains being a bit acausal, decision theory wise.
I thought anger and revenge (used in one shot situations) might be generalising from what to do in the iterated version which is what we had for more of our evolutionary history.
I kinda like a-causal decision theory for choosing to vote at all. I will choose to vote so that other people like me choose to vote.
It is the “learn to walk before you start learning to run and jump” approach.
Imagine that you are the first human who ever tried to invent math. You have invented the “2 + 2 = 4” theorem, and it seems to work! Two stones together with other two stones always make four stones. And two bananas together with other two bananas always make four bananas. Seems like you have found a powerful abstraction behind stones and bananas—who knows what else might be obeying the same rules. Excited you run towards your tribe members to share your discovery...
...but they dismiss it as worthless. “No two stones are same. A sharp stone is a more useful tool than a blunt stone, and your theory ignores that. A banana can be eaten. Actually, bananas are only useful for eating, and you completely ignore this essential part. And have you ever tried to ‘add’ two and two raindrops? Obviously, this ‘mathematics’ of yours has no future, and is just a waste of time” they say.
Well, the answer is that you need to apply the “2 + 2 = 4” only in the situations where it is appropriate, and not where it’s not. And telling the difference may be tricky. And it may be easy to underestimate some problem. But still, the equation is useful in some contexts, which makes it useful to know it.
Does refusing to know “2 + 2 = 4” because of these practical concerns make you stronger or weaker?
It’s an assumption to simplify the problem. First we learn to solve the “simple” problem, then we move to the more complex ones. If we can’t solve the simpler problem yet, would jumping to the more complex one make us progress faster?
Would a recommendation to skip solving the simplified problems, because they don’t include all the messy details, more likely lead to prople solving the hard problems faster, or giving up?
How specifically are you planning to use this insight in constructing a Friendly superintelligent AI?
2+2=4 pays rent.
Ugh can use it right away for counting days. Source code based decision theory not so much. There aren’t the societies based on agents that can read each others source code, so I can’t try and predict them with source code based decision theories. It seems like it is mathematically interesting thing though, so it is still interesting. I just don’t want it to be a core part of our sole pathway to try and solve the AI problem.
Perhaps it would lead people to avoid trying to find optimal decision theories and accept the answer that the best decision theory depends on the circumstances. Then we can figure out what our circumstances are and find good decisions theories for those. And create designs that can do similarly.
Like the best search algorithm is context dependent, where even algorithms of a worst complexity class can be better due to memory locality and small size.
Supposition> If answers to how decisions are made (and a whole host of other problems) are contextual and complex then it is worth trading information about what answers they have found within their context.
Figure the control flows between parts of the human brain that means they manage to roughly align themselves into a coherent entity (looking at dopamine etc).
Apply the same informational constraints to computers so that we can link a computer up to a human (it doesn’t need to be physically as long as the control flows work). The computer should be aligned with the user as much as a part of the brain is aligned to another part (which is hopefully sufficient).
While this is in an embryonic stage get as many people from all around the world to use it .
Hard take off (I think this less likely due to the context sensitivity things I have described, but still possible)- it is worthwhile to trade between agents so groups of human/computers that co-operate and trade information advance quicker. than loan rogue agents. If we have done the alignment work right then it is likely the weight of them will be able to squash any human inimical systems that appear.
This pathway makes no sense from people that expect there to be winner take all optimal AI designs as any one embryonic system might find the keys to the future and take it over. But if that is not the way the world works....
There are mathematical concepts that didn’t pay rent immediately: imaginary numbers, quaternions, non-Euclidean geometry, Turing machines...
But thanks for the specific plan, it sounds like it could work.
If you have any interest in my work into a hypothesis for 1 let me know.
Human beings can probabilistically read each others’ source code. That’s why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.
This seems like a weird way of say, humans can make/refine hypotheses about other agents. What does talking about source code give you?
Tit for tat (which seems like revenge) works in normal game theories for IPD (of infinite length) which is a closer to what we experience in everyday life. I thought Non-causal decision theories are needed for winning on one-shots?
In the case of humans, “talking about source code” is perhaps not that useful, though we do have source code, it’s written in quaternary and has a rather complex probabilistic compiler. And that source code was optimized by a purely causal process, demonstrating the fact that causal decision theory agents self modify into acausal decision theory agents in many circumstances.
Revenge and anger work for one shot problems, for example if some stranger comes and sexually assaults your wife, they cannot escape your wrath by “saying oh it’s only one shot, I’m a stranger in a huge city you’ll never see me again so there’s no point taking revenge”. You want to punch the in the face as an end in itself now, this is a simple way of our brains being a bit acausal, decision theory wise.
I thought anger and revenge (used in one shot situations) might be generalising from what to do in the iterated version which is what we had for more of our evolutionary history.
I kinda like a-causal decision theory for choosing to vote at all. I will choose to vote so that other people like me choose to vote.