I think infinite ethics will most likely be solved in a way that leaves longtermism unharmed.
Yes, or it might just never be truly “solved”. I agree that complexity theory seems fairly likely to hold (something like) a solution.
Do you have specific candidate solutions in mind?
Not really. I don’t think about infinite ethics much, which is probably one of the reasons it seems likely to change my mind. I expect that if I spent more time thinking about it, I would just become increasingly convinced that it isn’t worth thinking about.
But it definitely troubles me that I haven’t taken the time to really understand it, since I feel like I am in a similar epistemic state to ML researchers who dismiss Xrisk concerns and won’t take the time to engage with them.
I guess there’s maybe a disanalogy there, though, in that it seems like people who *have* thought more about infinite ethics tend to not be going around trying to convince others that it really actually matters a lot and should change what they work on or which causes they prioritize.
-------------------
I guess the main way I can imagine changing my views by studying infinite ethics would be to start believing that I should actually just aim to increase the chances of generating infinite utility (to the extent this is actually a mathematically coherent thing to try to do), which doesn’t necessarily/obviously lead to prioritizing Xrisk, as far as I can see.
The possibility of such an update seems like it might make studying infinite ethics until I understand it better a higher priority than reducing AI-Xrisk.
I guess there’s maybe a disanalogy there, though, in that it seems like people who have thought more about infinite ethics tend to not be going around trying to convince others that it really actually matters a lot and should change what they work on or which causes they prioritize.
Yep, seems like a good reason to not be too worried about it...
I guess the main way I can imagine changing my views by studying infinite ethics would be to start believing that I should actually just aim to increase the chances of generating infinite utility (to the extent this is actually a mathematically coherent thing to try to do), which doesn’t necessarily/obviously lead to prioritizing Xrisk, as far as I can see.
If infinite utility is actually possible, then not maximizing the chances of generating infinite utility would count as an x-risk, wouldn’t it? And it seems like the best way to prevent that would be to build a superintelligent AI that would do a good job of maximizing the chances of generating infinite utility, in case that was possible. Metaphilosophical AI seems to be an obvious approach to this.
And it seems like the best way to prevent that would be to build a superintelligent AI that would do a good job of maximizing the chances of generating infinite utility, in case that was possible.
I haven’t thought about it enough to say… it certainly seems plausible, but it seems plausible that spending a good chunk of time thinking about it *might* lead to different conclusions. *shrug
I think it makes sense to at least spend some time reading up on papers and posts about infinite ethics then. It doesn’t take very long to catch up to the state of the art and then you’ll probably have a much better idea if infinite ethics would be a better field to spend more time in, and it’s probably a good idea for an AI alignment researcher to have some background in it anyway. I’d say the same thing about moral uncertainty, if your reasoning about that is similar.
I have spent *some* time on it (on the order of 10-15hrs maybe? counting discussions, reading, etc.), and I have a vague intention to do so again, in the future. At the moment, though, I’m very focused on getting my PhD and trying to land a good professorship ~ASAP.
The genesis of this list is basically me repeatedly noticing that there are crucial considerations I’m ignoring (/more like procrastinating on :P) that I don’t feel like I have a good justification for ignoring, and being bothered by that.
It seemed important enough to at least *flag* these things.
If you think most AI alignment researchers should have some level of familiarity with these topics, it seems like it would be valuable for someone to put together a summary for us. I might be interested in such a project at some point in the next few years.
The genesis of this list is basically me repeatedly noticing that there are crucial considerations I’m ignoring (/more like procrastinating on :P) that I don’t feel like I have a good justification for ignoring, and being bothered by that.
It seemed important enough to at least flag these things.
That makes sense. Suggest putting this kind of background info in your future posts to give people more context.
If you think most AI alignment researchers should have some level of familiarity with these topics, it seems like it would be valuable for someone to put together a summary for us.
Hmm, I guess I think that more for moral uncertainty than for infinite ethics. For infinite ethics, it’s more that I think at least some people in AI alignment should have some level of familiarity, and it makes sense for whoever is most interested the topic (or otherwise motivated to learn it) to learn about it. Others could just have some sense of “this is a philosophical problem that may be relevant, I’ll look into it more in the future if I need to.”
I’m often prioritizing posting over polishing posts, for better or worse.
I’m also sometimes somewhat deliberately underspecific in my statements because I think it can lead to more interesting / diverse / “outside-the-box” kinds of responses that I think are very valuable from an “idea/perspective generation/exposure” point-of-view (and that’s something I find very valuable in general).
Yes, or it might just never be truly “solved”. I agree that complexity theory seems fairly likely to hold (something like) a solution.
Not really. I don’t think about infinite ethics much, which is probably one of the reasons it seems likely to change my mind. I expect that if I spent more time thinking about it, I would just become increasingly convinced that it isn’t worth thinking about.
But it definitely troubles me that I haven’t taken the time to really understand it, since I feel like I am in a similar epistemic state to ML researchers who dismiss Xrisk concerns and won’t take the time to engage with them.
I guess there’s maybe a disanalogy there, though, in that it seems like people who *have* thought more about infinite ethics tend to not be going around trying to convince others that it really actually matters a lot and should change what they work on or which causes they prioritize.
-------------------
I guess the main way I can imagine changing my views by studying infinite ethics would be to start believing that I should actually just aim to increase the chances of generating infinite utility (to the extent this is actually a mathematically coherent thing to try to do), which doesn’t necessarily/obviously lead to prioritizing Xrisk, as far as I can see.
The possibility of such an update seems like it might make studying infinite ethics until I understand it better a higher priority than reducing AI-Xrisk.
Yep, seems like a good reason to not be too worried about it...
If infinite utility is actually possible, then not maximizing the chances of generating infinite utility would count as an x-risk, wouldn’t it? And it seems like the best way to prevent that would be to build a superintelligent AI that would do a good job of maximizing the chances of generating infinite utility, in case that was possible. Metaphilosophical AI seems to be an obvious approach to this.
I haven’t thought about it enough to say… it certainly seems plausible, but it seems plausible that spending a good chunk of time thinking about it *might* lead to different conclusions. *shrug
I think it makes sense to at least spend some time reading up on papers and posts about infinite ethics then. It doesn’t take very long to catch up to the state of the art and then you’ll probably have a much better idea if infinite ethics would be a better field to spend more time in, and it’s probably a good idea for an AI alignment researcher to have some background in it anyway. I’d say the same thing about moral uncertainty, if your reasoning about that is similar.
I have spent *some* time on it (on the order of 10-15hrs maybe? counting discussions, reading, etc.), and I have a vague intention to do so again, in the future. At the moment, though, I’m very focused on getting my PhD and trying to land a good professorship ~ASAP.
The genesis of this list is basically me repeatedly noticing that there are crucial considerations I’m ignoring (/more like procrastinating on :P) that I don’t feel like I have a good justification for ignoring, and being bothered by that.
It seemed important enough to at least *flag* these things.
If you think most AI alignment researchers should have some level of familiarity with these topics, it seems like it would be valuable for someone to put together a summary for us. I might be interested in such a project at some point in the next few years.
That makes sense. Suggest putting this kind of background info in your future posts to give people more context.
Hmm, I guess I think that more for moral uncertainty than for infinite ethics. For infinite ethics, it’s more that I think at least some people in AI alignment should have some level of familiarity, and it makes sense for whoever is most interested the topic (or otherwise motivated to learn it) to learn about it. Others could just have some sense of “this is a philosophical problem that may be relevant, I’ll look into it more in the future if I need to.”
I’m often prioritizing posting over polishing posts, for better or worse.
I’m also sometimes somewhat deliberately underspecific in my statements because I think it can lead to more interesting / diverse / “outside-the-box” kinds of responses that I think are very valuable from an “idea/perspective generation/exposure” point-of-view (and that’s something I find very valuable in general).