One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
I said “one does not have to be convinced of the correctness of something in order to test it”, and you replied with something about usefulness. Therefore, there is nothing I can say about your response except that it’s utterly unrelated to what I said.
You are the one who introduced correctness into the argument. Alicorn said:
Do you expect anyone to benefit from your expertise if you can’t convince them you have it?
Feel free to read this as ‘convince them your expertise is “useful” ’ rather than your assumed ‘convince them your expertise is “correct” ’.
The underlying point is that there is a very large amount of apparently useless advice out there, and many self-help techniques seem initially useful but then stop being useful. (as you are well aware since your theory claims to explain why it happens)
The problem is to convince someone to try your advice, you have to convince them that the (probability of it being useful claimed benefit probability of claim being correct) is greater than the opportunity cost of the expected effort to try it. Due to others in the self-help market, the prior for it being useful is very low, and the prior for the claimed benefits equaling the actual benefits is low.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale. Dishonest salespeople spoil the sales possibilities for all the honest ones.
If you can convince someone with a higher standing in the community than you to test your advice and comment on the results of their test, you can raise individual’s probability expectations about the usefulness (or correctness) of your advice, and hence help more people than you otherwise would have.
P.S. I did go to your site and get added to your mailing list. However, even if your techniques turn out positively for me, I don’t think I have any higher standing in this community than you do, so I doubt my results will hold much weight with this group.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale.
Actually, I’m also running into a bias that merely because I have things to sell, I’m therefore trying to sell something in all places at all times… or that I’m always trying to “convince” people of something.
Indeed, the fact that you (and others) seem to think I need or even want to “convince” people of things is a symptom of this. Nobody goes around insisting that say, Yvain needs to get some high-status people to validate his ideas and “convince” the “community” to accept them!
If I had it all to do over again, I think I would have joined under a pseudonym and never let on I even had a business.
You are certainly right that “one does not have to be convinced of the correctness of something in order to test it.” But as you also said, immediately prior, “Either someone uses the information I give or they don’t.”
If we test information that we do not have reason to believe is useful, then we have a massive search space to cover. Much of the point of LW is to suggest useful regions for search, based on previous data.
So no, correctness is not a necessary condition of usefulness. But things that are correct are usually rather useful, and things that are not correct are less so. To the extent that you or your expertise are reliable indicators of the quality of your information, they help evaluate the probability of your information being useful, and hence the expected benefit of testing it.
Perhaps some parties on LW are actually confused by the distinction between truth and utility. I do not suspect Vladimir_Nesov is one of them.
But things that are correct are usually rather useful, and things that are not correct are less so.
Really? With what probability?
Or to put it another way: how were people were to start and put out fires for millennia before they had a correct theory of fire? Work metals without a correct atomic or molecular theory? Build catapults without a correct theory of gravity? Breed plants and animals without a correct theory of genetics?
In the entire history of humanity, “Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”, but there’s usually a ton more information you need besides the correct theory to get to “useful”, and more often, the theory ends up being derived from something that’s already “useful” anyway.
That’s a shockingly poor argument. Who can constrain the future more effectively: someone who knows the thermodynamics of combustion engines, or someone who only knows how to start fires with a flint-and-steel and how to stop them with water? Someone who can use X-ray crystallography to assess their metallurgy, or someone who has to whack their product with a mallet to see if it’s brittle? Someone who can fire mortars over ranges requiring Coriolis corrections (i.e., someone with a correct theory of mechanics) or someone who only knows how to aim a catapult by trial and error? Someone who can insert and delete bacterial genes, or someone who doesn’t even know germ theory?
Someone who actually knows how human cognition works on all scales, or someone with the equivalent of a set of flint-and-steel level tools and a devotion to trial and error?
‘Correctness’ in theories is a scalar rather than a binary quality. Phlogiston theory is less correct (and less useful) than chemistry, but it’s more correct—and more useful!--than the theory of elements. The fact that the modern scientific theories you list are better than their precursors, does not mean their precursors were useless.
You have a false dichotomy going here. If you know of someone who “knows how human cognition works on all scales”, or even just a theory of cognition as powerful as Newton’s theory of mechanics is in its domain, then please, link! But if such a theory existed, we wouldn’t need to be having this discussion. A strong theory of cognition will descend from a series of lesser theories of cognition, of which control theory is one step.
Unless you have a better theory, or a convincing reason to claim that “no-theory” is better than control theory, you’re in the position of an elementalist arguing that phlogiston theory should be ignored because it can’t explain heat generated by friction—while ignoring the fact that while imperfect, phlogiston theory is strictly superior to elemental theory or “no-theory”.
You’ve misunderstood my emphasis. I’m an engineer—I don’t insist on correctness. In each case I’ve picked above, the emphasis is on a deeper understanding (a continuous quantity, not a binary variable), not on truth per se. (I mention correctness in the Coriolis example, but even there I have Newtonian mechanics in mind, so that usage was not particularly accurate.)
My key perspective can be found in the third paragraph of this comment.
I’m all for control theory as a basis for forming hypotheses and for Seth Roberts-style self-experimentation.
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
That’s kind of confusing.
What I said was:
One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
And you gave an argument that some correct things are useful. Bravo.
However, you did not dispute the part where “useful” almost always comes before “correct”… thereby demonstrating precisely the confusion I posted about.
Useful and correct are not the same, and optimizing for correctness does not necessarily optimize usefulness, nor vice versa. That which is useful can be made correct, but that which is merely correct may be profoundly non-useful.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
(To the extent you would pick the latter, you likely hold an irrational bias… which would also explain the fanboy outrage and downvotes that my comments on this subject usually provoke here.)
I did not simply argue that some correct things are useful. I pointed out that every example of usefulness you presented can be augmented beyond all recognition with a deeper understanding of what is actually going on.
Let me put it this way: when you write, “how were people were to start and put out fires for millennia...” the key word is “start”: being satisfied with a method that works but provides no deep understanding is stagnation.
Ever seeking more useful methods without seeking to understand what is actually going on makes you an expert at whatever level of abstraction you’re stuck on. Order-of-magnitude advancement comes by improving the abstraction.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
I would also pick the former, provided my number one choice was not practical (perhaps due to time or resource constraints). The number one choice is to devote time and effort to making the true theory practicable. But if you never seek a true theory, you will never face this choice.
ETA: I’ll address:
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
by saying that you are arguing against, and I am arguing for:
But things that are correct are usually rather useful, and things that are not correct are less so.
I agree with Cyan, but even more basically, the set of correct beliefs necessarily includes any and all useful beliefs, because anything that is useful but incorrect can be derived from correct beliefs as well (similar to Eliezer’s Bayesians vs. Barbarians argument).
So, probabilistically, we should note that P(Useful|Correct)>P(Useful|Incorrect) because the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs. More importantly, as Sideways notes, more correct beliefs produce more useful effects; we don’t know now whether we have a “correct” theory of genetics, but it’s quite a bit more useful than its predecessor.
You still don’t get it. Correct beliefs don’t spring full-grown from the forehead of Omega—they come from observations. And to get observations, you have to be doing something… most likely, something useful.
That’s why your math is wrong for observed history—humans nearly always get “useful” first, then “correct”.
Or to put it another way, in theory you can get to practice from theory, but in practice, you almost never do.
Let’s assume that what you say is true, that utility precedes accuracy (and I happen to believe this is the case).
That does not in any way change the math. Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
It doesn’t matter if you have an Einstein’s grasp of the physical laws, a Ford’s grasp of the mechanics, and a lawyer’s mastery of traffic law… you still have to practice in order to learn to drive.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
This is why, when one defines “rationality” in terms of strictly abstract mentations and theoretical truths, one tends to lose in the “real world” to people who have actually practiced winning.
And I wasn’t arguing that definition, nor did I perceive any of the above discussion to be related to it. I’m arguing the relative utility of correct and incorrect beliefs, and the way in which the actual procedure of testing a position is related to the expected usefulness of that position.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
To the extent that my argument and the one you mention here interact, I suppose I would say that “winning” should include not just individual instances, things we can practice explicitly, but success in areas with which we are unfamiliar. That, I suggest, is the role of theory and the pursuit of correct beliefs.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong, but it seems to me that robotics has gradually progressed from having lots of complicated theories and sophisticated machinery towards simple control systems and improved sensory perception… and that this progression happened because the theories didn’t work in practice.
So, AFAICT, the argument that “if you have a correct theory, things will go better” is itself one of those ideas that work better in theory than in practice, because usually the only way to get a correct theory is to go out and try stuff.
Hindsight bias tends to make us completely ignore the fact that most discoveries come about from essentially random ideas and tinkering. We don’t like the idea that it’s not our “intelligence” that’s responsible, and we can very easily say that, in hindsight, the robotics theories were wrong, and of course if they had the right theory, they wouldn’t have made those mistakes.
But this is delusion. In theory, you could have a correct theory before any practice, but in practice, you virtually never do. (And pointing to nuclear physics as a counterexample is like pointing to lottery winners as proof that you can win the lottery; in theory, you can win the lottery, but in practice, you don’t.)
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong
You are wrong. The above is a myth promoted by the Culture of Chaos and the popular media. Advanced modern robots use advanced modern theory—e.g. particle filters to integrate multiple sensory streams to localize the robot (a Bayesian method).
And this is even more true when considering elements in the formation of a robot that need to be handled before the AI: physics, metallurgy, engineering, computer hardware design, etc.
Without theory—good, workably-correct theory—the search space for innovations is just too large. The more correct the theory, the less space has to be searched for solution concepts. If you’re going to build a rocket, you sure as hell better understand Newton’s laws. But things will go much smoother if you also know some chemistry, some material science, and some computer science.
For a solid example of theory taking previous experimental data and massively narrowing the search space, see RAND’s first report on the feasibility of satellites here.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
Procedures are brittle. Theory lets you generalize procedures for new contexts, which you can then practice.
the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs.
I’m not sure I’d grant that unless you can show it mathematically. It seems to me there are infinite beliefs of all sorts, and I’m not sure how their orders compare.
Select an arbitrary true predicate sentence Rab. That sentence almost certainly (in the mathematical sense) is false if an arbitrary c is substituted for b. Thus, whatever the cardinality of the set of true sentences, for every true sentence we can construct infinitely many false sentences, where the opposite is not true. Thus, the cardinality of the set of true sentences is greater than the set of false sentences.
I don’t think that’s as rigorous as you’d like it to be. I don’t grant the “almost certainly false” step.
Take a predicate P which is false for Pab but true in all other cases. Then, you cannot perform the rest of the steps in your proof with P. Consider that there is also the predicate Q such that Qab is true about half the time for arbitrary a and b. How will you show that most situations are like your R?
I’m also not sure your proof really shows a difference in cardinality. Even if most predicates are like your R, there still might be infinitely many true sentences you can construct, even if they’re more likely to be false.
It’s definitely not rigorous, and I tried to highlight that by calling it a heuristic. Without omniscience, I can’t prove that the relations hold, but the evidence is uniformly supportive.
Can you name such a predicate other than the trivial “is not” (which is guaranteed for be true for all but one entity, as in A is not A) which is true for even a majority of entities? The best I can do is “is not describable by a message of under N bits,” but even then there are self-referential issues. If the majority of predicates were like your P and Q, then why would intelligence be interesting? “Correctness” would be the default state of a proposition and we’d only be eliminating a (relatively) small number of false hypotheses from our massive pool of true ones. Does that match either your experience or the more extensive treatment provided in Eliezer’s writings on AI?
If you grant my assertion that Rab is almost certainly false if c is substituted for b, then I think the cardinality proof does follow. Since we cannot put the true sentences in one-to-one correspondence with the false sentences, and by the assertion there are more false sentences, the latter must have a greater (infinite?) cardinality than the former, no?
You’re right. I was considering constructive statements, since the negation of an arbitrary false statement has infinitesimal informational value in search, but you’re clearly right when considering all statements.
If by “almost certainly false” you mean that say, 1 out of every 10,000 such sentences will be true, then no, that does not entail a higher order of infinity.
I meant, as in the math case, that the probability of selecting a true statement by choosing one at random out of the space of all possible statements is 0 (there are true statements, but as a literal infinitesimal).
It’s possible that both infinities are countable, as I am not sure how one would prove it either way, but that detail doesn’t really matter for the broader argument.
See the note by JGWeissman—this is only true when considering constructively true statements (those that carry non-negligible informational content, i.e. not the negation of an arbitrary false statement).
“Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”,
Which is it?
I think all the further you can go with this line of thought is to point out that lots of things are useful even if we don’t have a correct theory for how they work. We have other ways to guess that something might be useful and worth trying.
Having a correct theory is always nice, but I don’t see that our choice here is between having a correct theory or not having one.
Perhaps surprisingly, statistics has an answer, and that answer is no. If in your application the usefulness of a statistical model is equivalent to its predictive performance, then choose your model using cross-validation, which directly optimizes for predictive performance. When that gets too expensive, use the AIC, which is equivalent to cross-validation as the amount of data grows without bound. But if the true model is available, neither AIC nor cross-validation will pick it out of the set of models being considered as the amount of data grows without bound.
define: A theory’s “truthfulness” as how much probability mass it has after appropriate selection of prior and applications of Bayes’ theorem. It works as a good measure for a theory’s “usefulness” as long as resource limitations and psychological side effects aren’t important.
define: A theory’s “usefulness” as a function of resources needed to calculate its predictions to a certain degree of accuracy, the “truthfulness” of the theory itself, and side effects. Squinting at it, I get something roughly like:
usefulness(truthfulness, resources, side effects) = truthfulness * accuracy(resources) + messiness(side effects)
So I define “usefulness” as a function and “truthfulness” as its limiting value as side effects go to 0 and resources go to infinity.
Notice how I shaped the definition of “usefulness” to avoid mention of context specific utilities; I purposefully avoided making it domain specific or talking about what the theory is trying to predict. I did this to maintain generality.
(Note: For now I’m polishing over the issue of how to deal with abstracting over concrete hypotheses and integrating the properties of this abstraction with the definitions)
Your definition of usefulness fails to include the utility of the predictions made, which is the most important factor. A theory is useful if there is a chain of inference from it to a concrete application, and its degree of usefulness depends on the utility of that application, whether it could have been reached without using the theory, and the resources required to follow that chain of inference. Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
From pwno:
“Aren’t true theories defined by how useful they are in some application?”
My definition of “usefulness” was built with the express purpose of relating the truth of theories to how useful they are and is very much a context specific temporary definition (hence “define:”). If I had tried to deal with it directly I would have had something uselessly messy and incomplete, or I could have used a true but also uninformative expectation approach and hid all of the complexity. Instead, I experimented and tried to force the concepts to unify in some way. To do so I stretched the definition of usefulness pretty much to the breaking point and omitted any direct relation to utility functions. I found it a useful thought to think and hope you do as well even if you take issue with my use of the name “usefulness”.
Actions of high utility are useful. Of a set of available actions, the correct action to select is the most useful one. A correct statement is one expressing the truth, or probabilistically, an event of high probability. In this sense, a correct choice of action is one of which it is a correct statement to say that it is the most useful one.
It’s beside the point actually, since you haven’t shown that your info is either useful or correct.
Do you expect anyone to benefit from your expertise if you can’t convince them you have it?
Either someone uses the information I give or they don’t. One does not have to be “convinced” of the correctness of something in order to test it.
But whether someone uses the information or not, what do I or my “expertise” have to do with it?
Someone is more likely to spend the time and effort to test something if they think it’s more likely to be correct.
It’s irrational of people who aren’t convinced that the information is useful to use it.
Either a tiger eats celery or it doesn’t. But the tiger has to be “convinced” that celery is tasty in order to taste it.
One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
I said “one does not have to be convinced of the correctness of something in order to test it”, and you replied with something about usefulness. Therefore, there is nothing I can say about your response except that it’s utterly unrelated to what I said.
You are the one who introduced correctness into the argument. Alicorn said:
Feel free to read this as ‘convince them your expertise is “useful” ’ rather than your assumed ‘convince them your expertise is “correct” ’.
The underlying point is that there is a very large amount of apparently useless advice out there, and many self-help techniques seem initially useful but then stop being useful. (as you are well aware since your theory claims to explain why it happens)
The problem is to convince someone to try your advice, you have to convince them that the (probability of it being useful claimed benefit probability of claim being correct) is greater than the opportunity cost of the expected effort to try it. Due to others in the self-help market, the prior for it being useful is very low, and the prior for the claimed benefits equaling the actual benefits is low.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale. Dishonest salespeople spoil the sales possibilities for all the honest ones.
If you can convince someone with a higher standing in the community than you to test your advice and comment on the results of their test, you can raise individual’s probability expectations about the usefulness (or correctness) of your advice, and hence help more people than you otherwise would have.
P.S. I did go to your site and get added to your mailing list. However, even if your techniques turn out positively for me, I don’t think I have any higher standing in this community than you do, so I doubt my results will hold much weight with this group.
Actually, I’m also running into a bias that merely because I have things to sell, I’m therefore trying to sell something in all places at all times… or that I’m always trying to “convince” people of something.
Indeed, the fact that you (and others) seem to think I need or even want to “convince” people of things is a symptom of this. Nobody goes around insisting that say, Yvain needs to get some high-status people to validate his ideas and “convince” the “community” to accept them!
If I had it all to do over again, I think I would have joined under a pseudonym and never let on I even had a business.
You are certainly right that “one does not have to be convinced of the correctness of something in order to test it.” But as you also said, immediately prior, “Either someone uses the information I give or they don’t.”
If we test information that we do not have reason to believe is useful, then we have a massive search space to cover. Much of the point of LW is to suggest useful regions for search, based on previous data.
So no, correctness is not a necessary condition of usefulness. But things that are correct are usually rather useful, and things that are not correct are less so. To the extent that you or your expertise are reliable indicators of the quality of your information, they help evaluate the probability of your information being useful, and hence the expected benefit of testing it.
Perhaps some parties on LW are actually confused by the distinction between truth and utility. I do not suspect Vladimir_Nesov is one of them.
Really? With what probability?
Or to put it another way: how were people were to start and put out fires for millennia before they had a correct theory of fire? Work metals without a correct atomic or molecular theory? Build catapults without a correct theory of gravity? Breed plants and animals without a correct theory of genetics?
In the entire history of humanity, “Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”, but there’s usually a ton more information you need besides the correct theory to get to “useful”, and more often, the theory ends up being derived from something that’s already “useful” anyway.
That’s a shockingly poor argument. Who can constrain the future more effectively: someone who knows the thermodynamics of combustion engines, or someone who only knows how to start fires with a flint-and-steel and how to stop them with water? Someone who can use X-ray crystallography to assess their metallurgy, or someone who has to whack their product with a mallet to see if it’s brittle? Someone who can fire mortars over ranges requiring Coriolis corrections (i.e., someone with a correct theory of mechanics) or someone who only knows how to aim a catapult by trial and error? Someone who can insert and delete bacterial genes, or someone who doesn’t even know germ theory?
Someone who actually knows how human cognition works on all scales, or someone with the equivalent of a set of flint-and-steel level tools and a devotion to trial and error?
‘Correctness’ in theories is a scalar rather than a binary quality. Phlogiston theory is less correct (and less useful) than chemistry, but it’s more correct—and more useful!--than the theory of elements. The fact that the modern scientific theories you list are better than their precursors, does not mean their precursors were useless.
You have a false dichotomy going here. If you know of someone who “knows how human cognition works on all scales”, or even just a theory of cognition as powerful as Newton’s theory of mechanics is in its domain, then please, link! But if such a theory existed, we wouldn’t need to be having this discussion. A strong theory of cognition will descend from a series of lesser theories of cognition, of which control theory is one step.
Unless you have a better theory, or a convincing reason to claim that “no-theory” is better than control theory, you’re in the position of an elementalist arguing that phlogiston theory should be ignored because it can’t explain heat generated by friction—while ignoring the fact that while imperfect, phlogiston theory is strictly superior to elemental theory or “no-theory”.
You’ve misunderstood my emphasis. I’m an engineer—I don’t insist on correctness. In each case I’ve picked above, the emphasis is on a deeper understanding (a continuous quantity, not a binary variable), not on truth per se. (I mention correctness in the Coriolis example, but even there I have Newtonian mechanics in mind, so that usage was not particularly accurate.)
My key perspective can be found in the third paragraph of this comment.
I’m all for control theory as a basis for forming hypotheses and for Seth Roberts-style self-experimentation.
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
That’s kind of confusing.
What I said was:
And you gave an argument that some correct things are useful. Bravo.
However, you did not dispute the part where “useful” almost always comes before “correct”… thereby demonstrating precisely the confusion I posted about.
Useful and correct are not the same, and optimizing for correctness does not necessarily optimize usefulness, nor vice versa. That which is useful can be made correct, but that which is merely correct may be profoundly non-useful.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
(To the extent you would pick the latter, you likely hold an irrational bias… which would also explain the fanboy outrage and downvotes that my comments on this subject usually provoke here.)
I did not simply argue that some correct things are useful. I pointed out that every example of usefulness you presented can be augmented beyond all recognition with a deeper understanding of what is actually going on.
Let me put it this way: when you write, “how were people were to start and put out fires for millennia...” the key word is “start”: being satisfied with a method that works but provides no deep understanding is stagnation.
Ever seeking more useful methods without seeking to understand what is actually going on makes you an expert at whatever level of abstraction you’re stuck on. Order-of-magnitude advancement comes by improving the abstraction.
I would also pick the former, provided my number one choice was not practical (perhaps due to time or resource constraints). The number one choice is to devote time and effort to making the true theory practicable. But if you never seek a true theory, you will never face this choice.
ETA: I’ll address:
by saying that you are arguing against, and I am arguing for:
Deep theory has profound long-term impact, but is useless for simple stuff.
What is considered simple stuff is itself a function of that profound long-term impact.
I agree with Cyan, but even more basically, the set of correct beliefs necessarily includes any and all useful beliefs, because anything that is useful but incorrect can be derived from correct beliefs as well (similar to Eliezer’s Bayesians vs. Barbarians argument).
So, probabilistically, we should note that P(Useful|Correct)>P(Useful|Incorrect) because the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs. More importantly, as Sideways notes, more correct beliefs produce more useful effects; we don’t know now whether we have a “correct” theory of genetics, but it’s quite a bit more useful than its predecessor.
You still don’t get it. Correct beliefs don’t spring full-grown from the forehead of Omega—they come from observations. And to get observations, you have to be doing something… most likely, something useful.
That’s why your math is wrong for observed history—humans nearly always get “useful” first, then “correct”.
Or to put it another way, in theory you can get to practice from theory, but in practice, you almost never do.
Let’s assume that what you say is true, that utility precedes accuracy (and I happen to believe this is the case).
That does not in any way change the math. Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
It doesn’t matter if you have an Einstein’s grasp of the physical laws, a Ford’s grasp of the mechanics, and a lawyer’s mastery of traffic law… you still have to practice in order to learn to drive.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
This is why, when one defines “rationality” in terms of strictly abstract mentations and theoretical truths, one tends to lose in the “real world” to people who have actually practiced winning.
And I wasn’t arguing that definition, nor did I perceive any of the above discussion to be related to it. I’m arguing the relative utility of correct and incorrect beliefs, and the way in which the actual procedure of testing a position is related to the expected usefulness of that position.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
To the extent that my argument and the one you mention here interact, I suppose I would say that “winning” should include not just individual instances, things we can practice explicitly, but success in areas with which we are unfamiliar. That, I suggest, is the role of theory and the pursuit of correct beliefs.
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong, but it seems to me that robotics has gradually progressed from having lots of complicated theories and sophisticated machinery towards simple control systems and improved sensory perception… and that this progression happened because the theories didn’t work in practice.
So, AFAICT, the argument that “if you have a correct theory, things will go better” is itself one of those ideas that work better in theory than in practice, because usually the only way to get a correct theory is to go out and try stuff.
Hindsight bias tends to make us completely ignore the fact that most discoveries come about from essentially random ideas and tinkering. We don’t like the idea that it’s not our “intelligence” that’s responsible, and we can very easily say that, in hindsight, the robotics theories were wrong, and of course if they had the right theory, they wouldn’t have made those mistakes.
But this is delusion. In theory, you could have a correct theory before any practice, but in practice, you virtually never do. (And pointing to nuclear physics as a counterexample is like pointing to lottery winners as proof that you can win the lottery; in theory, you can win the lottery, but in practice, you don’t.)
You are wrong. The above is a myth promoted by the Culture of Chaos and the popular media. Advanced modern robots use advanced modern theory—e.g. particle filters to integrate multiple sensory streams to localize the robot (a Bayesian method).
And this is even more true when considering elements in the formation of a robot that need to be handled before the AI: physics, metallurgy, engineering, computer hardware design, etc.
Without theory—good, workably-correct theory—the search space for innovations is just too large. The more correct the theory, the less space has to be searched for solution concepts. If you’re going to build a rocket, you sure as hell better understand Newton’s laws. But things will go much smoother if you also know some chemistry, some material science, and some computer science.
For a solid example of theory taking previous experimental data and massively narrowing the search space, see RAND’s first report on the feasibility of satellites here.
IAWYC but
Procedures are brittle. Theory lets you generalize procedures for new contexts, which you can then practice.
I’m not sure I’d grant that unless you can show it mathematically. It seems to me there are infinite beliefs of all sorts, and I’m not sure how their orders compare.
A heuristic method that underlies my reasoning:
Select an arbitrary true predicate sentence Rab. That sentence almost certainly (in the mathematical sense) is false if an arbitrary c is substituted for b. Thus, whatever the cardinality of the set of true sentences, for every true sentence we can construct infinitely many false sentences, where the opposite is not true. Thus, the cardinality of the set of true sentences is greater than the set of false sentences.
I don’t think that’s as rigorous as you’d like it to be. I don’t grant the “almost certainly false” step.
Take a predicate P which is false for Pab but true in all other cases. Then, you cannot perform the rest of the steps in your proof with P. Consider that there is also the predicate Q such that Qab is true about half the time for arbitrary a and b. How will you show that most situations are like your R?
I’m also not sure your proof really shows a difference in cardinality. Even if most predicates are like your R, there still might be infinitely many true sentences you can construct, even if they’re more likely to be false.
It’s definitely not rigorous, and I tried to highlight that by calling it a heuristic. Without omniscience, I can’t prove that the relations hold, but the evidence is uniformly supportive.
Can you name such a predicate other than the trivial “is not” (which is guaranteed for be true for all but one entity, as in A is not A) which is true for even a majority of entities? The best I can do is “is not describable by a message of under N bits,” but even then there are self-referential issues. If the majority of predicates were like your P and Q, then why would intelligence be interesting? “Correctness” would be the default state of a proposition and we’d only be eliminating a (relatively) small number of false hypotheses from our massive pool of true ones. Does that match either your experience or the more extensive treatment provided in Eliezer’s writings on AI?
If you grant my assertion that Rab is almost certainly false if c is substituted for b, then I think the cardinality proof does follow. Since we cannot put the true sentences in one-to-one correspondence with the false sentences, and by the assertion there are more false sentences, the latter must have a greater (infinite?) cardinality than the former, no?
The cardinality of the sets of true and false statements is the same. The operation of negation is a bijection between them.
You’re right. I was considering constructive statements, since the negation of an arbitrary false statement has infinitesimal informational value in search, but you’re clearly right when considering all statements.
If by “almost certainly false” you mean that say, 1 out of every 10,000 such sentences will be true, then no, that does not entail a higher order of infinity.
I meant, as in the math case, that the probability of selecting a true statement by choosing one at random out of the space of all possible statements is 0 (there are true statements, but as a literal infinitesimal).
It’s possible that both infinities are countable, as I am not sure how one would prove it either way, but that detail doesn’t really matter for the broader argument.
See the note by JGWeissman—this is only true when considering constructively true statements (those that carry non-negligible informational content, i.e. not the negation of an arbitrary false statement).
Which is it?
I think all the further you can go with this line of thought is to point out that lots of things are useful even if we don’t have a correct theory for how they work. We have other ways to guess that something might be useful and worth trying.
Having a correct theory is always nice, but I don’t see that our choice here is between having a correct theory or not having one.
Both. Over the course of history:
Useful things → mostly not true theories.
True theory → usually useful, but mostly first preceded by useful w/untrue theory.
Aren’t true theories defined by how useful they are in some application?
Perhaps surprisingly, statistics has an answer, and that answer is no. If in your application the usefulness of a statistical model is equivalent to its predictive performance, then choose your model using cross-validation, which directly optimizes for predictive performance. When that gets too expensive, use the AIC, which is equivalent to cross-validation as the amount of data grows without bound. But if the true model is available, neither AIC nor cross-validation will pick it out of the set of models being considered as the amount of data grows without bound.
define: A theory’s “truthfulness” as how much probability mass it has after appropriate selection of prior and applications of Bayes’ theorem. It works as a good measure for a theory’s “usefulness” as long as resource limitations and psychological side effects aren’t important.
define: A theory’s “usefulness” as a function of resources needed to calculate its predictions to a certain degree of accuracy, the “truthfulness” of the theory itself, and side effects. Squinting at it, I get something roughly like: usefulness(truthfulness, resources, side effects) = truthfulness * accuracy(resources) + messiness(side effects)
So I define “usefulness” as a function and “truthfulness” as its limiting value as side effects go to 0 and resources go to infinity. Notice how I shaped the definition of “usefulness” to avoid mention of context specific utilities; I purposefully avoided making it domain specific or talking about what the theory is trying to predict. I did this to maintain generality.
(Note: For now I’m polishing over the issue of how to deal with abstracting over concrete hypotheses and integrating the properties of this abstraction with the definitions)
Your definition of usefulness fails to include the utility of the predictions made, which is the most important factor. A theory is useful if there is a chain of inference from it to a concrete application, and its degree of usefulness depends on the utility of that application, whether it could have been reached without using the theory, and the resources required to follow that chain of inference. Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
Thank you—that’s an excellent summary.
From pwno: “Aren’t true theories defined by how useful they are in some application?”
My definition of “usefulness” was built with the express purpose of relating the truth of theories to how useful they are and is very much a context specific temporary definition (hence “define:”). If I had tried to deal with it directly I would have had something uselessly messy and incomplete, or I could have used a true but also uninformative expectation approach and hid all of the complexity. Instead, I experimented and tried to force the concepts to unify in some way. To do so I stretched the definition of usefulness pretty much to the breaking point and omitted any direct relation to utility functions. I found it a useful thought to think and hope you do as well even if you take issue with my use of the name “usefulness”.
Actions of high utility are useful. Of a set of available actions, the correct action to select is the most useful one. A correct statement is one expressing the truth, or probabilistically, an event of high probability. In this sense, a correct choice of action is one of which it is a correct statement to say that it is the most useful one.
It’s beside the point actually, since you haven’t shown that your info is either useful or correct.