This is an interesting way to frame things. I have plenty of experience what you’re calling aspiration here via deliberative practices over the past 5 years or so that have caused me to transform in ways I wanted to while also not understanding how to get there. For example, when I started zen practice I had some vague idea of what I was there to do or get—get “enlightened”, be more present, be more capable, act more naturally, etc.—but I didn’t really understand how to do it or even what it was I was really going for. After all, if I really did understand it, I would have already been doing it. It’s only through a very slow process of experimenting, trying, being nudged in directions, and making very short moves towards nearby attractors that I’ve over time come to better unstand some of these things, or understand why I was confused and what the thing I thought I wanted really was without being skewed by my previous perceptions of it.
I think much of the problem with the kind of approach you are proposing is figuring out how to turn this into something a machine can do. That is, right now it’s understood and explained at a level that makes sense for humans, but how do we take those notions and turn them into something mathematically precise enough that we could instruct a machine to do them and then evaluate whether or not what it did was in fact what we intended. I realize you are just pointing out the idea and not claiming to have it all solved, so this is only to say that I expect much of the hard work here is figuring out what the core, natural feature of what’s going on with aspiration is such that it can be used to design an AI that can do that.
how do we take those notions and turn them into something mathematically precise enough that we could instruct a machine to do them and then evaluate whether or not what it did was in fact what we intended
Yep, that’s the project! I think the main utility of Callard’s work here is (1) pointing out the phenomenon (a phenomenon that is strikingly similar to some of the abilities we want AI’s to have), and (2) noticing that the most prominent theories of decision theory, moral psychology, and moral responsibility make assumptions that we have to break if we want to allow room for aspiration (assumptions that we who are trying to build safe AI are probably also accidentally making insofar as we take over those standard theories). IDK whether she provides alternate assumptions to make instead, but if she does these might also be useful. But the main point is just noticing that we need different theories of these things.
Once we’ve noticed the phenomenon of aspiration, and that it requires breaking some of these assumptions, I agree that the hard bit is coming up with a mathematical theory of aspiration (or the AI equivalent).
This is an interesting way to frame things. I have plenty of experience what you’re calling aspiration here via deliberative practices over the past 5 years or so that have caused me to transform in ways I wanted to while also not understanding how to get there. For example, when I started zen practice I had some vague idea of what I was there to do or get—get “enlightened”, be more present, be more capable, act more naturally, etc.—but I didn’t really understand how to do it or even what it was I was really going for. After all, if I really did understand it, I would have already been doing it. It’s only through a very slow process of experimenting, trying, being nudged in directions, and making very short moves towards nearby attractors that I’ve over time come to better unstand some of these things, or understand why I was confused and what the thing I thought I wanted really was without being skewed by my previous perceptions of it.
I think much of the problem with the kind of approach you are proposing is figuring out how to turn this into something a machine can do. That is, right now it’s understood and explained at a level that makes sense for humans, but how do we take those notions and turn them into something mathematically precise enough that we could instruct a machine to do them and then evaluate whether or not what it did was in fact what we intended. I realize you are just pointing out the idea and not claiming to have it all solved, so this is only to say that I expect much of the hard work here is figuring out what the core, natural feature of what’s going on with aspiration is such that it can be used to design an AI that can do that.
Yep, that’s the project! I think the main utility of Callard’s work here is (1) pointing out the phenomenon (a phenomenon that is strikingly similar to some of the abilities we want AI’s to have), and (2) noticing that the most prominent theories of decision theory, moral psychology, and moral responsibility make assumptions that we have to break if we want to allow room for aspiration (assumptions that we who are trying to build safe AI are probably also accidentally making insofar as we take over those standard theories). IDK whether she provides alternate assumptions to make instead, but if she does these might also be useful. But the main point is just noticing that we need different theories of these things.
Once we’ve noticed the phenomenon of aspiration, and that it requires breaking some of these assumptions, I agree that the hard bit is coming up with a mathematical theory of aspiration (or the AI equivalent).