I’m probably not “aligned” in a way that generalizes to having dangerous superpowers, uncertain personhood and rights, purposefully limited perspective, and somewhere between thousands to billions of agents trying to manipulate and exploit me for their own purposes. I expect even a self-modified Best Extrapolated Version of me would struggle gravely with doing well by other beings in this situation. Cultish attractor basins are hazards for even the most benign set of values for humans, and a highly-controlled situation with a lot of dangerous influence like that might exacerbate that particular risk.
But I do believe that hypothetical self-modifying has at least the potential to help me Do Better, because doing better is often a skills issue, learning skills is a currently accessible form of self-modification with good results, and self-modifying might help with learning skills.
I’m probably not “aligned” in a way that generalizes to having dangerous superpowers, uncertain personhood and rights, purposefully limited perspective, and somewhere between thousands to billions of agents trying to manipulate and exploit me for their own purposes. I expect even a self-modified Best Extrapolated Version of me would struggle gravely with doing well by other beings in this situation. Cultish attractor basins are hazards for even the most benign set of values for humans, and a highly-controlled situation with a lot of dangerous influence like that might exacerbate that particular risk.
But I do believe that hypothetical self-modifying has at least the potential to help me Do Better, because doing better is often a skills issue, learning skills is a currently accessible form of self-modification with good results, and self-modifying might help with learning skills.