I think someone pursuing a mathematics education PhD is a perfect example for Jeff’s X/Y/Z tweet (which I agree with).
I think there’s a very good chance that that will still be a valuable thing for her to pursue
But how valuable? (It’s easy to focus on how it’s very likely more than zero value, but that would be missing the point.) Is the marginal expected value really anywhere close to the marginal EV of raising the probability of a pause in the next couple years?
Sadly, I think not.
This can be really hard to admit for emotional reasons, especially if she is passionately focused on this path and reluctant to consider radical changes to unusual paths like working to pause AI.
And of course, since pausing AI is the path you’re pursuing, then even if you admitted it (or took the view—I don’t want to use too much language that presumes I’m right), might this just lead to unnecessery tension in the relationship? “Hey romantic parter, I think that your expected social impact would be much higher if you switched from your current path which you’re very passionate about to working on what I’m working on instead” seems pretty awkward TBH. I’d expect most people in that situation to feel like “Why is my partner trying to discourage me from doing what I’m passionate about?” and “Aren’t you biased to think that what you’re doing is most valuable?” and feel bad like they aren’t being supported. If they aren’t unusually open to pivoting, they probably won’t pivot, and so will end up having the same impact that they would have had if you hadn’t said anything, but will just think that you don’t think what they’re doing is important. And that sucks for both of you. It would be much nicer if it could just be true that continuing with her current path is the most valuable thing she can do.
Like most people, she isn’t strictly focused on maximizing expected value. (Which is fine! It’s okay to be human in ordinary human ways! That’s part of what I’m fighting to protect.) She seems to accept the arguments for extinction risk, but then compartmentalizes hard, and commits to being agnostic, saying “I don’t understand those claims” about anything at the existential level.
She listens to me all the time and has been very supportive of my activism, so I haven’t pushed for her to have more involvement, since I’m fortunate to have this level of stability and support in the first place.
As a side note, her knowledge of constructivism and other math education topics means that a lot of what we have learned about artificial minds resonates with what she has learned about human minds. Math education research was established by mathematicians moreso than teachers, so it is extremely technical and more like cognitive neuroscience than pedagogy. I’d bring her the latest in my understandings of deep learning theory and interpretability, rambling on about the nature of understanding and intelligence, then say, “I probably sound crazy.” And she’d say, “No, you sound like a constructivist.”
I feel you—this does hit hard emotionally.
I think someone pursuing a mathematics education PhD is a perfect example for Jeff’s X/Y/Z tweet (which I agree with).
But how valuable? (It’s easy to focus on how it’s very likely more than zero value, but that would be missing the point.) Is the marginal expected value really anywhere close to the marginal EV of raising the probability of a pause in the next couple years?
Sadly, I think not.
This can be really hard to admit for emotional reasons, especially if she is passionately focused on this path and reluctant to consider radical changes to unusual paths like working to pause AI.
And of course, since pausing AI is the path you’re pursuing, then even if you admitted it (or took the view—I don’t want to use too much language that presumes I’m right), might this just lead to unnecessery tension in the relationship? “Hey romantic parter, I think that your expected social impact would be much higher if you switched from your current path which you’re very passionate about to working on what I’m working on instead” seems pretty awkward TBH. I’d expect most people in that situation to feel like “Why is my partner trying to discourage me from doing what I’m passionate about?” and “Aren’t you biased to think that what you’re doing is most valuable?” and feel bad like they aren’t being supported. If they aren’t unusually open to pivoting, they probably won’t pivot, and so will end up having the same impact that they would have had if you hadn’t said anything, but will just think that you don’t think what they’re doing is important. And that sucks for both of you. It would be much nicer if it could just be true that continuing with her current path is the most valuable thing she can do.
Yep, that’s the situation.
Like most people, she isn’t strictly focused on maximizing expected value. (Which is fine! It’s okay to be human in ordinary human ways! That’s part of what I’m fighting to protect.) She seems to accept the arguments for extinction risk, but then compartmentalizes hard, and commits to being agnostic, saying “I don’t understand those claims” about anything at the existential level.
She listens to me all the time and has been very supportive of my activism, so I haven’t pushed for her to have more involvement, since I’m fortunate to have this level of stability and support in the first place.
As a side note, her knowledge of constructivism and other math education topics means that a lot of what we have learned about artificial minds resonates with what she has learned about human minds. Math education research was established by mathematicians moreso than teachers, so it is extremely technical and more like cognitive neuroscience than pedagogy. I’d bring her the latest in my understandings of deep learning theory and interpretability, rambling on about the nature of understanding and intelligence, then say, “I probably sound crazy.” And she’d say, “No, you sound like a constructivist.”