I guess I was unclear. I mean that basing decisions upon comparison of some real numbers is pretty silly, the real numbers being the action utilities. One could instead compare trees and effectively stop branching when it is clear enough one tree is larger than other. This also provides a way to eliminate bias due to one tree being pruned more than the other.
The world utilities too are expensive to calculate for the worlds that are hard to simulate.
The world utilities too are expensive to calculate for the worlds that are hard to simulate.
So, to recap, the way this is supposed to work is that organisms predict their future sense-data using inductive inference. They don’t predict their entire future world, just their own future perceptions of it. Their utility function then becomes their own projected future happiness. All intelligent agents do something like this. The cost of simulating the universe they are in thus turns out to be a big non-issue.
Predicting future sense requires simulation of a fairly big chunk of the world compared to the agent itself.
We don’t do that, even. We take two trees, and evaluate them together comparing as we go, so that we don’t need to sum values of things that are not different between the two, and don’t need to branch off into identical branches. This way we evaluate change caused by an action—the difference between worlds—rather than the worlds. One can quit evaluation once one is sufficiently certain that difference >0 or difference<0 .
One thing we certainly don’t do when we are comparing two alternatives, is coming up with a number for one, then coming up with a number for another, then comparing. No, we write list of pros and cons of each, side to side, to ensure non biased comparison and to ensure we can stop working earlier.
Predicting future sense requires simulation of a fairly big chunk of the world compared to the agent itself.
We don’t do that, even [...]
Sure we do. Our minds are constantly predicting the future. We predict the future and then update our predictions (and discard inaccurate ones) when surprising sense data comes in. The predictions cover practically all sense data. That’s how we know when our model is wrong when encountering surprising sense data—since we have already made our predictions.
We take two trees, and evaluate them together comparing as we go, so that we don’t need to sum values of things that are not different between the two, and don’t need to branch off into identical branches. This way we evaluate change caused by an action—the difference between worlds—rather than the worlds.
Well, hang on. I’m not saying humans don’t optimise this type of task! Humans are a cobbled-together, unmaintainable mess, though. Surely they illustrate how NOT to build a mind. There are all kinds of drawbacks to only calculating relative utilities—for one you can’t easily store them and compare their utility with those of other courses of action. Is it even worth doing? I do not know—which is why I propose profiling before optimising.
I meant, we don’t so much predict ‘the future world’ as the changes to it, to cut on the amount that we need to simulate.
Is it even worth doing? I do not know—which is why I propose profiling before optimising.
What if I know? I am a software developer. I propose less expensive method for deciding on the algorithmic optimizations: learn from existing software such as chess AIs (which are packed with algorithmic optimizations).
edit: also, you won’t learn from profiling that high level optimization is worth doing. Suppose you write a program that eliminates duplicate entries from a file, and you did it the naive way: comparing each to each, O(n^2) . You may find out via profiling whenever most of the time is spent reading the entries, or comparing them, and you may spend time optimizing those, but you won’t learn that you can sort entries first to eliminate the duplicates efficiently. Same goes for things like e.g. raytracers in computer graphics. Practical example from a programming contest: the contestants had 10 seconds to render image with a lot of light reflection inside ellipsoids (the goal was accuracy of output). The reference image was done using straightforward photon mapping—randomly shot photons from light sources—run over time of ~10 hours. The noise is proportional to 1/sqrt(n) ; it converges slowly. The top contestants, myself included, fired photons in organized patterns; the result converged as 1/n . The n being way large even in single second, the contestants did beat the contest organizer’s reference image by far. It would of took months for the contest organizers solution to beat result of contestants in 10 seconds. (edit: the contest sort of failed in result though because the only way to rank images was to compare them to contest organizer’s solution)
The profiler—well, sure, the contest organizers could of ran profiler instead of ‘optimizing prematurely’, and could of found out that their refraction is where they spent most time (or ray ellipsoid intersection or whatever else), and they could of optimized those, for unimportant speed gain. The truth is, they did not even know that their method was too slow, without seeing the superior method (they wouldn’t even have thought so if told, nor could have been convinced with the reasoning that the contestants had used to determine the method to use).
I guess I was unclear. I mean that basing decisions upon comparison of some real numbers is pretty silly, the real numbers being the action utilities. One could instead compare trees and effectively stop branching when it is clear enough one tree is larger than other. This also provides a way to eliminate bias due to one tree being pruned more than the other.
The world utilities too are expensive to calculate for the worlds that are hard to simulate.
So, to recap, the way this is supposed to work is that organisms predict their future sense-data using inductive inference. They don’t predict their entire future world, just their own future perceptions of it. Their utility function then becomes their own projected future happiness. All intelligent agents do something like this. The cost of simulating the universe they are in thus turns out to be a big non-issue.
Predicting future sense requires simulation of a fairly big chunk of the world compared to the agent itself.
We don’t do that, even. We take two trees, and evaluate them together comparing as we go, so that we don’t need to sum values of things that are not different between the two, and don’t need to branch off into identical branches. This way we evaluate change caused by an action—the difference between worlds—rather than the worlds. One can quit evaluation once one is sufficiently certain that difference >0 or difference<0 .
One thing we certainly don’t do when we are comparing two alternatives, is coming up with a number for one, then coming up with a number for another, then comparing. No, we write list of pros and cons of each, side to side, to ensure non biased comparison and to ensure we can stop working earlier.
Sure we do. Our minds are constantly predicting the future. We predict the future and then update our predictions (and discard inaccurate ones) when surprising sense data comes in. The predictions cover practically all sense data. That’s how we know when our model is wrong when encountering surprising sense data—since we have already made our predictions.
Well, hang on. I’m not saying humans don’t optimise this type of task! Humans are a cobbled-together, unmaintainable mess, though. Surely they illustrate how NOT to build a mind. There are all kinds of drawbacks to only calculating relative utilities—for one you can’t easily store them and compare their utility with those of other courses of action. Is it even worth doing? I do not know—which is why I propose profiling before optimising.
I meant, we don’t so much predict ‘the future world’ as the changes to it, to cut on the amount that we need to simulate.
What if I know? I am a software developer. I propose less expensive method for deciding on the algorithmic optimizations: learn from existing software such as chess AIs (which are packed with algorithmic optimizations).
edit: also, you won’t learn from profiling that high level optimization is worth doing. Suppose you write a program that eliminates duplicate entries from a file, and you did it the naive way: comparing each to each, O(n^2) . You may find out via profiling whenever most of the time is spent reading the entries, or comparing them, and you may spend time optimizing those, but you won’t learn that you can sort entries first to eliminate the duplicates efficiently. Same goes for things like e.g. raytracers in computer graphics. Practical example from a programming contest: the contestants had 10 seconds to render image with a lot of light reflection inside ellipsoids (the goal was accuracy of output). The reference image was done using straightforward photon mapping—randomly shot photons from light sources—run over time of ~10 hours. The noise is proportional to 1/sqrt(n) ; it converges slowly. The top contestants, myself included, fired photons in organized patterns; the result converged as 1/n . The n being way large even in single second, the contestants did beat the contest organizer’s reference image by far. It would of took months for the contest organizers solution to beat result of contestants in 10 seconds. (edit: the contest sort of failed in result though because the only way to rank images was to compare them to contest organizer’s solution)
The profiler—well, sure, the contest organizers could of ran profiler instead of ‘optimizing prematurely’, and could of found out that their refraction is where they spent most time (or ray ellipsoid intersection or whatever else), and they could of optimized those, for unimportant speed gain. The truth is, they did not even know that their method was too slow, without seeing the superior method (they wouldn’t even have thought so if told, nor could have been convinced with the reasoning that the contestants had used to determine the method to use).