On one hand, yes, many things that seem like “bugs” in short term, start seeming like “features” when you consider the second-order effects, etc.
On the other hand, some people update to the opposite extreme, and conclude that there are no “bugs”; that every behavior, no matter how seemingly random, short-sighted, or self-destructive, is a result of some hidden deeper wisdom that we merely do not understand yet.
I call this the just world fallacy, the proponents prefer to talk about revealed preferences. The idea is that everything that you do, or everything that merely happens to you, is by definition something you (perhaps unconsciously) wanted to happen. So in the end, everyone gets exactly what they wanted most, exactly in the way they wanted it most, there is no problem to solve, ever. Of course, it sounds stupid when I put it so bluntly, but it can be further defended by saying “okay, even if you say that X happened by accident, there was certainly a way to prevent X or at least reduce its probability, if only you spent enough resources towards that goal, but it was your choice—your revealed preference—to spend those resources on something else instead, so at least in that sense you preferred X to sometimes happen, statistically”.
So I would recommend caution in both directions. Don’t call things “bugs” in thinking, just because you cannot think about any useful purpose in 5 seconds. On the other hand, do not conclude that humanity has a revealed preference to be converted to paperclips, and we are merely in denial about it (most likely for signalling reasons).
I think you’re contrasting ideas that don’t contradict each other.
(1) On the other hand, some people update to the opposite extreme, and conclude that there are no “bugs”; that every behavior, no matter how seemingly random, short-sighted, or self-destructive, is a result of some hidden deeper wisdom that we merely do not understand yet.
(2) The idea is that everything that you do, or everything that merely happens to you, is by definition something you (perhaps unconsciously) wanted to happen. So in the end, everyone gets exactly what they wanted most, exactly in the way they wanted it most, there is no problem to solve, ever.
I believe in 1, but not in 2 (not 100%). Something may contain “a deeper wisdom” and be unsuccessful in the real world. It’s true even for such wisdom as “rationality”: one day you may find out that we live in a simulation and everyone who practiced rationality was punished. But it won’t make rationality “silly”. I think “wisdom” is what we want to call it, what we decide to call it.
I think a rationalist may connect 1 and 2 because “the problem to solve” is very important for rationality (on a philosophical level). Anything that diminishes its importance looks the same. So, “we need to find places where our wisdom fits the world” or “if you failed you may’ve chosen a bad goal” sounds like “there’re no problems ever, we don’t need to solve any problems ever”.
So in the end, everyone gets exactly what they wanted most, exactly in the way they wanted it most, there is no problem to solve, ever.
On the other hand, do not conclude that humanity has a revealed preference to be converted to paperclips, and we are merely in denial about it (most likely for signalling reasons).
It’s a strange way to put it, but I think you can put it like this if you consider multiple possible worlds.
For example, you could say a person who behaves recklessly has a preference to suffer the risks of reckless behavior for the chance to live her life fully in a world where reckless behavior makes sense. I think such formulation may reveal more about people ignoring AI risks (I don’t support ignoring those risks) than “people can’t understand risks, people can’t calculate”.
If we assume that behavior has a genetical component, and that mutations are random, than would imply that some behaviors we observe are at least partially random. Do you disagree with this?
If that randomness in the behavior happens to be harmful, would it be okay to call it a “bug”?
I may dislike my decisions or habits. But to understand if it’s a “bug” or not I would need to have a near complete understanding of my own thinking. I don’t think I have that. So I don’t see any goal/gain of conceptualizing something as a “bug”. If my behavior depends on genes or weather or anything else it’s not relevant for me at the moment.
On one hand, yes, many things that seem like “bugs” in short term, start seeming like “features” when you consider the second-order effects, etc.
On the other hand, some people update to the opposite extreme, and conclude that there are no “bugs”; that every behavior, no matter how seemingly random, short-sighted, or self-destructive, is a result of some hidden deeper wisdom that we merely do not understand yet.
I call this the just world fallacy, the proponents prefer to talk about revealed preferences. The idea is that everything that you do, or everything that merely happens to you, is by definition something you (perhaps unconsciously) wanted to happen. So in the end, everyone gets exactly what they wanted most, exactly in the way they wanted it most, there is no problem to solve, ever. Of course, it sounds stupid when I put it so bluntly, but it can be further defended by saying “okay, even if you say that X happened by accident, there was certainly a way to prevent X or at least reduce its probability, if only you spent enough resources towards that goal, but it was your choice—your revealed preference—to spend those resources on something else instead, so at least in that sense you preferred X to sometimes happen, statistically”.
So I would recommend caution in both directions. Don’t call things “bugs” in thinking, just because you cannot think about any useful purpose in 5 seconds. On the other hand, do not conclude that humanity has a revealed preference to be converted to paperclips, and we are merely in denial about it (most likely for signalling reasons).
I think you’re contrasting ideas that don’t contradict each other.
I believe in 1, but not in 2 (not 100%). Something may contain “a deeper wisdom” and be unsuccessful in the real world. It’s true even for such wisdom as “rationality”: one day you may find out that we live in a simulation and everyone who practiced rationality was punished. But it won’t make rationality “silly”. I think “wisdom” is what we want to call it, what we decide to call it.
I think a rationalist may connect 1 and 2 because “the problem to solve” is very important for rationality (on a philosophical level). Anything that diminishes its importance looks the same. So, “we need to find places where our wisdom fits the world” or “if you failed you may’ve chosen a bad goal” sounds like “there’re no problems ever, we don’t need to solve any problems ever”.
It’s a strange way to put it, but I think you can put it like this if you consider multiple possible worlds.
For example, you could say a person who behaves recklessly has a preference to suffer the risks of reckless behavior for the chance to live her life fully in a world where reckless behavior makes sense. I think such formulation may reveal more about people ignoring AI risks (I don’t support ignoring those risks) than “people can’t understand risks, people can’t calculate”.
If we assume that behavior has a genetical component, and that mutations are random, than would imply that some behaviors we observe are at least partially random. Do you disagree with this?
If that randomness in the behavior happens to be harmful, would it be okay to call it a “bug”?
I’m not sure it would change much for me.
I may dislike my decisions or habits. But to understand if it’s a “bug” or not I would need to have a near complete understanding of my own thinking. I don’t think I have that. So I don’t see any goal/gain of conceptualizing something as a “bug”. If my behavior depends on genes or weather or anything else it’s not relevant for me at the moment.