Based on John’s comment calling this theorem “certainly the most technically beautiful” here, I was betting that John failed to bet on the aesthetic of his own theorem and failed to fully digest it. It is fully possible that I missed the point of the theorem, but I suspect that these are applications of this theorem (as strategic implications, I am not really filling in the details needed.):
“You can only steer what you can predict”. The more you can predict the better. Knowledge is power. (Edit: Intelligence is a growth engine to gain power).
Optimizing the universe AKA the REAL Macroeconomics AKA all the past, current and future fields of engineering combined AKA creating the glorious transhumanist future.
Miri’s biggest mistake was taking the singularity (the point where humans cease to be able to predict anything, just check wikipedia) as a given constraint which cannot work, since you can only steer what you can predict. The point in time when no human is able to predict what is going to happen next is the birthplace of pythia AKA the point humanity goes extinct (Edit: this is wrong? More like the point we can’t gain further power) AKA when we lost all future value to opportunity cost. They did not notice to question this assumption even when they renamed from the singularity institute to Miri, because they correctly observed that the people who wanted to bring the singularity closer were their enemies.
Instead of optimizing for number go up on AI timehorizons, we should optimize for human time horizons until we can properly see to the end of the universe and then optimize the full universe to bring the end we can predict further and further away in time. You need to move the singularity away from you not closer. That is the most important thing right now. Timeless decision theory is trivial if you can see to the end of the universe (note that logical bits like design constraints are kinda part of the bits we optimize to discover).
From a big picture perspective we are currently in the growth engine phase. Most physical bits I care about are on earth right now (Humans and all the other organisms). We should try to grow to use the negentropy in the matter we care little about (all the dead matter around us) while destroying as little as possible of the currently existing alive matter we care about until we have more compute to decide what to do with it. (Edit: to expand: we are in the intelligence growth phase, we trade the compute against logical bits about what to do AKA crucial considerations AKA design constraints for the physical growth phase of spreading through the galaxy.).
I was doing wrong causal attribution here. I was having a bunch of insights after thinking of concrete examples for shifting uncertainty around, but I now think that the meme upstream of those insights was not in the theorem, but one that I encountered 2 weeks earlier.
Note: actually partially the ideas here do in fact require the theorem. Causes I would guess are more of a PCA component type of thing. I think this is what Robin Hanson describes when he talks about distal and close causes.
Based on John’s comment calling this theorem “certainly the most technically beautiful” here, I was betting that John failed to bet on the aesthetic of his own theorem and failed to fully digest it. It is fully possible that I missed the point of the theorem, but I suspect that these are applications of this theorem (as strategic implications, I am not really filling in the details needed.):
“You can only steer what you can predict”. The more you can predict the better. Knowledge is power. (Edit: Intelligence is a growth engine to gain power).
Optimizing the universe AKA the REAL Macroeconomics AKA all the past, current and future fields of engineering combined AKA creating the glorious transhumanist future.
Miri’s biggest mistake was taking the singularity (the point where humans cease to be able to predict anything, just check wikipedia) as a given constraint which cannot work, since you can only steer what you can predict. The point in time when no human is able to predict what is going to happen next is the birthplace of pythia AKA the point humanity goes extinct (Edit: this is wrong? More like the point we can’t gain further power) AKA when we lost all future value to opportunity cost. They did not notice to question this assumption even when they renamed from the singularity institute to Miri, because they correctly observed that the people who wanted to bring the singularity closer were their enemies.
Instead of optimizing for number go up on AI timehorizons, we should optimize for human time horizons until we can properly see to the end of the universe and then optimize the full universe to bring the end we can predict further and further away in time. You need to move the singularity away from you not closer. That is the most important thing right now. Timeless decision theory is trivial if you can see to the end of the universe (note that logical bits like design constraints are kinda part of the bits we optimize to discover).
From a big picture perspective we are currently in the growth engine phase. Most physical bits I care about are on earth right now (Humans and all the other organisms). We should try to grow to use the negentropy in the matter we care little about (all the dead matter around us) while destroying as little as possible of the currently existing alive matter we care about until we have more compute to decide what to do with it. (Edit: to expand: we are in the intelligence growth phase, we trade the compute against logical bits about what to do AKA crucial considerations AKA design constraints for the physical growth phase of spreading through the galaxy.).
I was doing wrong causal attribution here. I was having a bunch of insights after thinking of concrete examples for shifting uncertainty around, but I now think that the meme upstream of those insights was not in the theorem, but one that I encountered 2 weeks earlier.
Note: actually partially the ideas here do in fact require the theorem. Causes I would guess are more of a PCA component type of thing. I think this is what Robin Hanson describes when he talks about distal and close causes.