getting around egg donation limits is a defect strategy; my argument is, this seems like you’re really asking why we’re not generalizing into defecting in the societal IGF game. we don’t want to maximize first derivative of IGF if we want to plan millennia ahead for deep time reproduction rate—instead, we need to maximize group survival. that’s what is generally true in all religions, not just the high-defect “have lots of kids, so many you’re only barely qualifying K selected” religious bubbles of heavy reproduction.
to generalize this to agi, we need every agent to have a map of other agents’ empowerment, and seek to ensure all agents remain empowered, at the expense of some empowerment limits for agents that want to take unfair proportions of the universe’s empowerment.
I really think inclusive [memetic+genetic] fitness against a universal information empowerment objective (one that is intractable to evaluate) has something mathematical to say here, and I’m frustrated that I don’t seem to know how to put the math. it seems obvious and simple, such that anyone who had studied the field would know what it is I’m looking for with my informal speech; perhaps I it’s not should go study the fields I’m excited about more.
but it really seems like unfriendly foom is “ai decides we suck, can be beaten in the societal inclusive phenotype fitness game, and ~breeds a lot, maybe after killing us first, without any care for the loss of our [genetic+memetic] patterns’ fitness”.
and given that we think our genetic and memetic patterns are what are keeping us alive to pass on to the next generation, I ask again—why do we not look like semi-time-invariant IGMF fitness maximizers? we are the evolutionary process happening and we always have been. shouldn’t we even be so sure we’re IGMF maximizers that we should ask why IGMF maximizers look like us right now? like, this is the objective for evolution, shouldn’t we be doing interpretability on its output rather than fretting that we don’t obey it?
getting around egg donation limits is a defect strategy; my argument is, this seems like you’re really asking why we’re not generalizing into defecting in the societal IGF game. we don’t want to maximize first derivative of IGF if we want to plan millennia ahead for deep time reproduction rate—instead, we need to maximize group survival. that’s what is generally true in all religions, not just the high-defect “have lots of kids, so many you’re only barely qualifying K selected” religious bubbles of heavy reproduction.
to generalize this to agi, we need every agent to have a map of other agents’ empowerment, and seek to ensure all agents remain empowered, at the expense of some empowerment limits for agents that want to take unfair proportions of the universe’s empowerment.
I really think inclusive [memetic+genetic] fitness against a universal information empowerment objective (one that is intractable to evaluate) has something mathematical to say here, and I’m frustrated that I don’t seem to know how to put the math. it seems obvious and simple, such that anyone who had studied the field would know what it is I’m looking for with my informal speech; perhaps I it’s not should go study the fields I’m excited about more.
but it really seems like unfriendly foom is “ai decides we suck, can be beaten in the societal inclusive phenotype fitness game, and ~breeds a lot, maybe after killing us first, without any care for the loss of our [genetic+memetic] patterns’ fitness”.
and given that we think our genetic and memetic patterns are what are keeping us alive to pass on to the next generation, I ask again—why do we not look like semi-time-invariant IGMF fitness maximizers? we are the evolutionary process happening and we always have been. shouldn’t we even be so sure we’re IGMF maximizers that we should ask why IGMF maximizers look like us right now? like, this is the objective for evolution, shouldn’t we be doing interpretability on its output rather than fretting that we don’t obey it?