I think that at the time this post came out, I didn’t have the mental scaffolding necessary to really engage with it – I thought of this question as maybe important, but sort of “above my paygrade”, something better left to other people who would have the resources to engage more seriously with it.
But, over the past couple years, the concepts here have formed an important component of my understanding of robust agency. Much of this came from private in-person conversations, but this post is the best writeup of the concept I’m currently aware of.
One thing I like about this post is the focus on philosophical competence. Previously, I’d thought of this question as dangerous to think about, because you might make philosophical mistakes that doomed you or your universe for (in retrospect) silly reasons.
My current model is more like “no, you-with-your-21st-century-human-brain shouldn’t actually attempt to take actions aiming primarily to affect other universes on the macro scale. Negotiating with other universes is something you do when you’re a literal galaxy brain that is quite confident in it’s philosophy.”
But, meanwhile, it seems that:
(note: low-to-mid confidence, still working through these problems myself, and I very much still philosophically confused about at least some of this)
– becoming philosophically competent, as a species, may be one of the most important goals facing humanity, and how this project interfaces with AI development may be crucially important. This may be relevant to people (like me) who aren’t directly working on alignment but trying to have a good model of the strategic landscape.
– a concept not from this particular post, but relevant, is the notion that the question is not “are you in a simulation or not?”, it’s more like “to what degree are you in simulations? Which distribution of agents are you making choices on behalf of?”. And this has some implications about how you should make choices that you should worry about now, before you are a literal galaxy brain. (many of which are more mundane)
– I think there may be a connection between “Beyond Astronominal Waste” and “Robustness to scale.” You can’t be a galaxy brain now, but you can be the sort of person who would be demonstrably safe to scale up, in a way that simulations can detect, which might let you punch above your current weight, in terms of philosophical competence.
I think that at the time this post came out, I didn’t have the mental scaffolding necessary to really engage with it – I thought of this question as maybe important, but sort of “above my paygrade”, something better left to other people who would have the resources to engage more seriously with it.
But, over the past couple years, the concepts here have formed an important component of my understanding of robust agency. Much of this came from private in-person conversations, but this post is the best writeup of the concept I’m currently aware of.
One thing I like about this post is the focus on philosophical competence. Previously, I’d thought of this question as dangerous to think about, because you might make philosophical mistakes that doomed you or your universe for (in retrospect) silly reasons.
My current model is more like “no, you-with-your-21st-century-human-brain shouldn’t actually attempt to take actions aiming primarily to affect other universes on the macro scale. Negotiating with other universes is something you do when you’re a literal galaxy brain that is quite confident in it’s philosophy.”
But, meanwhile, it seems that:
(note: low-to-mid confidence, still working through these problems myself, and I very much still philosophically confused about at least some of this)
– becoming philosophically competent, as a species, may be one of the most important goals facing humanity, and how this project interfaces with AI development may be crucially important. This may be relevant to people (like me) who aren’t directly working on alignment but trying to have a good model of the strategic landscape.
– a concept not from this particular post, but relevant, is the notion that the question is not “are you in a simulation or not?”, it’s more like “to what degree are you in simulations? Which distribution of agents are you making choices on behalf of?”. And this has some implications about how you should make choices that you should worry about now, before you are a literal galaxy brain. (many of which are more mundane)
– I think there may be a connection between “Beyond Astronominal Waste” and “Robustness to scale.” You can’t be a galaxy brain now, but you can be the sort of person who would be demonstrably safe to scale up, in a way that simulations can detect, which might let you punch above your current weight, in terms of philosophical competence.