Yea, there is often an emphasis on how an ideal unconstrained agent (with a resource-unlimited AIXI like superintelligence in mind) would reason, or how biases should be avoided, with the assumption that a resource-limited agent should just try to approximate what a (practically) resource-unlimited agent should do as closely as it can given its resources.
That, however, is a bad assumption to make. For every limitation-class, an entirely different strategy and different heuristics may be optimal.
As an example, when you try to program your old calculator to play speed chess as well as possible, you should not try to take the best available chess program. There are biases you should adopt to get the best result, given the limitations. I was once in a grad AI class in which we needed to best a given computer opponent in an obscure game, with severe computational resource limitations. The best program was a seemingly bad hack of in generality wrong heuristics, but it turned out to use the resources in the most optimal way. It bested much “better” programs that would’ve beaten it if given more time.
Take teaching the Bayes equation as if that’s what you’d actually use. Sure, some general ideas (consider the prior, always update on your observations) are useful, but the equation itself? Noone at CFAR walks into a supermarket and then continuously inputs actual numbers into Bayes equations in their head.
Take teaching the Bayes equation as if that’s what you’d actually use. Sure, some general ideas (consider the prior, always update on your observations) are useful, but the equation itself? Noone at CFAR walks into a supermarket and then continuously inputs actual numbers into Bayes equations in their head.
Agreed! I teach the Bayes class at workshops, and it’s not a math drill class. It’s on how to get the habit of paying attention to the components of Bayes theorem in everyday life. For example, we usually ask just ask “Would I be likely to see Y if X were true?” and skip the question “Would I be likely to see Y if X were not true?” So we practice ways to trigger this thought so you don’t get tricked by base rates or other pitfalls.
Concrete example: Someone you’re interviewing for a job flubs one question and your first thought is that your shouldn’t hire them, because people who aren’t qualified flub questions. But pause and ask how often you expect qualified people to miss one question in an hour long interview. Your answer will vary based on the kinds of questions your asking, but you may be treated the evidence as a stronger signal than it is.
Concrete example: Someone you’re interviewing for a job flubs one question and your first thought is that your shouldn’t hire them, because people who aren’t qualified flub questions. But pause and ask how often you expect qualified people to miss one question in an hour long interview. Your answer will vary based on the kinds of questions your asking, but you may be treated the evidence as a stronger signal than it is.
Or maybe you treated it as a weaker signal than it is. This is a strawman anyway, people who never in their life heard of Bayes do compare it to their hypothetical idea of how a competent person would do on their exam, and soon thereafter, to their actual knowledge of how a competent person does, remedying all sorts of miscalibrations.
If anything, in practice the actual problem with interviews is generally that incompetents get through, because incompetents are being interviewed so much more than anyone else. Diligent ability to never flunk anything (conscientiousness) is, at least, something very useful in workplace that you can’t fake by preparing specifically for interviews.
Then there’s also this enormous utility disparity between the minor dis-utility of perhaps running the interviews for a little longer and ending up hiring the best when no one passes, and major dis-utility of hiring an incompetent.
It’s not really an advice, but I can see how its likeable—there’s people who didn’t get hired because they flunked “maybe one question”, and these folks will get a fix of their endorphins when they rationalize it as the HR person being irrational.
More generally, a very good test will result in a uniform distribution of scores (rather than a bell curve), maximizing the information content of the score.
Well, even bigger issue with Bayes is how easy it is to get it wrong on graphs in general (which contain loops). Worse than that, what we actually have is uncertain topology. This all should make rational updates much smaller and compartmentalized-looking than some naive idea of ‘updating’ beliefs from one argument, then from the other, and so on.
There’s also a lot of very advanced math related to specifically estimation. E.g. an expected utility would be a huge sum, vast majority of terms in which you do not know. When deciding on some binary action, you have those two sums on both sides of comparison, and you need to estimate the sign of the difference most accurately (which is dramatically not same as summing all available terms), then you need to quantify the expected inaccuracy in your estimate of the sign, and adjust for that. Simply put, its complicated and people who have good working understanding of such concerns can write important textbooks, software, papers, etc. (Which make a lot of difference to the world, as well as make any spin off ‘workshops’ credible). Whereas people who are very far from having any understanding of such can do things like estimating 8 lives saved per dollar donated.
Yea, there is often an emphasis on how an ideal unconstrained agent (with a resource-unlimited AIXI like superintelligence in mind) would reason, or how biases should be avoided, with the assumption that a resource-limited agent should just try to approximate what a (practically) resource-unlimited agent should do as closely as it can given its resources.
That, however, is a bad assumption to make. For every limitation-class, an entirely different strategy and different heuristics may be optimal.
As an example, when you try to program your old calculator to play speed chess as well as possible, you should not try to take the best available chess program. There are biases you should adopt to get the best result, given the limitations. I was once in a grad AI class in which we needed to best a given computer opponent in an obscure game, with severe computational resource limitations. The best program was a seemingly bad hack of in generality wrong heuristics, but it turned out to use the resources in the most optimal way. It bested much “better” programs that would’ve beaten it if given more time.
Take teaching the Bayes equation as if that’s what you’d actually use. Sure, some general ideas (consider the prior, always update on your observations) are useful, but the equation itself? Noone at CFAR walks into a supermarket and then continuously inputs actual numbers into Bayes equations in their head.
Agreed! I teach the Bayes class at workshops, and it’s not a math drill class. It’s on how to get the habit of paying attention to the components of Bayes theorem in everyday life. For example, we usually ask just ask “Would I be likely to see Y if X were true?” and skip the question “Would I be likely to see Y if X were not true?” So we practice ways to trigger this thought so you don’t get tricked by base rates or other pitfalls.
Concrete example: Someone you’re interviewing for a job flubs one question and your first thought is that your shouldn’t hire them, because people who aren’t qualified flub questions. But pause and ask how often you expect qualified people to miss one question in an hour long interview. Your answer will vary based on the kinds of questions your asking, but you may be treated the evidence as a stronger signal than it is.
Or maybe you treated it as a weaker signal than it is. This is a strawman anyway, people who never in their life heard of Bayes do compare it to their hypothetical idea of how a competent person would do on their exam, and soon thereafter, to their actual knowledge of how a competent person does, remedying all sorts of miscalibrations.
If anything, in practice the actual problem with interviews is generally that incompetents get through, because incompetents are being interviewed so much more than anyone else. Diligent ability to never flunk anything (conscientiousness) is, at least, something very useful in workplace that you can’t fake by preparing specifically for interviews.
Then there’s also this enormous utility disparity between the minor dis-utility of perhaps running the interviews for a little longer and ending up hiring the best when no one passes, and major dis-utility of hiring an incompetent.
It’s not really an advice, but I can see how its likeable—there’s people who didn’t get hired because they flunked “maybe one question”, and these folks will get a fix of their endorphins when they rationalize it as the HR person being irrational.
A good test will include a few questions almost no-one can answer. That avoids the problem of having more than one 10⁄10 score.
More generally, a very good test will result in a uniform distribution of scores (rather than a bell curve), maximizing the information content of the score.
Well, even bigger issue with Bayes is how easy it is to get it wrong on graphs in general (which contain loops). Worse than that, what we actually have is uncertain topology. This all should make rational updates much smaller and compartmentalized-looking than some naive idea of ‘updating’ beliefs from one argument, then from the other, and so on.
There’s also a lot of very advanced math related to specifically estimation. E.g. an expected utility would be a huge sum, vast majority of terms in which you do not know. When deciding on some binary action, you have those two sums on both sides of comparison, and you need to estimate the sign of the difference most accurately (which is dramatically not same as summing all available terms), then you need to quantify the expected inaccuracy in your estimate of the sign, and adjust for that. Simply put, its complicated and people who have good working understanding of such concerns can write important textbooks, software, papers, etc. (Which make a lot of difference to the world, as well as make any spin off ‘workshops’ credible). Whereas people who are very far from having any understanding of such can do things like estimating 8 lives saved per dollar donated.