I found this essay frustrating. Heavy use of metaphor in philosophy is fine as long as it’s grounded out or paid off; this didn’t get there.
Edit: Okay, I went back and reread the entire sequence. it did get places, but in very low ratio to its length. And the overall message between the many lines in this particular post, “green is something and it’s cool”, is not one that’s well-argued or particularly useful. Is it cool, or does it just feel cool sometimes? Joe doesn’t claim to know, and I don’t either. And I’m still not sure green is a coherent thing worth having a single name for (John Wentworth’s hypothesis of “oxytocin” in his review is very interesting).
After rereading the whole sequence, I now have much more to say. I am confused as to why this post and the sequence ranked so high in people’s votes for year’s best. Did people not already believe all of this? Were they hungry for some poetry and intuition to back the rationalist project? I thought Yudkowsky already provided that. But I did enjoy Joe’s tracking over the same ground with a different sort of eloquence and evocation. I’m just not sure I’d recommend it widely, since it took so long to read/listen, and is not really very idea/logic dense. It is in some sense a non-rationalist explication of the standard rationalist philosophy.
When I saw that my review was pretty prominent publicly, I went back and reread the piece to make sure I wasn’t being unfair. Then I found I had to reread the entire sequence to make sure, since perhaps the payoff was elsewhere.
I love ethical philosophy as much as the next rationalist; I’ve spent plenty of time on it myself. I haven’t spent that time recently nor written much about it, because I think its importance (like most topics) is dwarfed by the immediacy of the alignment problem.
This series claims to be relevant by noting that we should not lock in the future prematurely. This is very true. But that point could be made much more directly.
I was not unfair. The writing is exquisite. But good writing without a sharp payoff is a mixed blessing. It takes time that’s in short supply.
Perhaps it’s good that these posts exist, as a sort of elaborate double-check of the Deep Athiest or Yudkowskian rationalist views. Joe actually lands rather near where he started, essentially in agreement IMO with Yud and what I take to be the centroid of rationalist views on ethics (including my own). He does raise some valid questions. Those questions could’ve been raised in a few paragraphs instead of a near-book length.
So: read them instead of watching a movie, or listen to them instead of a different six hours of entertainment. If these posts were among the best of the year for you, I’m glad you’re now caught up on the deep atheist/rationalist logic.
Here’s how I’d boil down the central points:
Respect what you don’t understand. Don’t discard it prematurely. Intuition may be a valid guide to your values.
And in firm agreement with Yudkowsky:
Love nature but don’t trust her. Be gentle once you hold enough power to protect what you love. Be open to finding new things to love.
One thought I took away was that human values are not a fixed set; they are generated by our DNA but they emerge in a complex interplay with the world. You have no “true heart”, only what you’ve built so far. So you and others might want to keep an open mind about finding new values without overwriting the old.
I hope to write more about this, but I do find technical alignment more pressing, and I still think that developers are more likely to pursue intent alignment, and to do something vaguely like a long reflection if we make it past the critical risk period. We have time for these important questions, if we have the wisdom to take it.
I found this essay frustrating. Heavy use of metaphor in philosophy is fine as long as it’s grounded out or paid off; this didn’t get there.
Edit: Okay, I went back and reread the entire sequence. it did get places, but in very low ratio to its length. And the overall message between the many lines in this particular post, “green is something and it’s cool”, is not one that’s well-argued or particularly useful. Is it cool, or does it just feel cool sometimes? Joe doesn’t claim to know, and I don’t either. And I’m still not sure green is a coherent thing worth having a single name for (John Wentworth’s hypothesis of “oxytocin” in his review is very interesting).
After rereading the whole sequence, I now have much more to say. I am confused as to why this post and the sequence ranked so high in people’s votes for year’s best. Did people not already believe all of this? Were they hungry for some poetry and intuition to back the rationalist project? I thought Yudkowsky already provided that. But I did enjoy Joe’s tracking over the same ground with a different sort of eloquence and evocation. I’m just not sure I’d recommend it widely, since it took so long to read/listen, and is not really very idea/logic dense. It is in some sense a non-rationalist explication of the standard rationalist philosophy.
When I saw that my review was pretty prominent publicly, I went back and reread the piece to make sure I wasn’t being unfair. Then I found I had to reread the entire sequence to make sure, since perhaps the payoff was elsewhere.
I love ethical philosophy as much as the next rationalist; I’ve spent plenty of time on it myself. I haven’t spent that time recently nor written much about it, because I think its importance (like most topics) is dwarfed by the immediacy of the alignment problem.
This series claims to be relevant by noting that we should not lock in the future prematurely. This is very true. But that point could be made much more directly.
I was not unfair. The writing is exquisite. But good writing without a sharp payoff is a mixed blessing. It takes time that’s in short supply.
Perhaps it’s good that these posts exist, as a sort of elaborate double-check of the Deep Athiest or Yudkowskian rationalist views. Joe actually lands rather near where he started, essentially in agreement IMO with Yud and what I take to be the centroid of rationalist views on ethics (including my own). He does raise some valid questions. Those questions could’ve been raised in a few paragraphs instead of a near-book length.
So: read them instead of watching a movie, or listen to them instead of a different six hours of entertainment. If these posts were among the best of the year for you, I’m glad you’re now caught up on the deep atheist/rationalist logic.
Here’s how I’d boil down the central points:
Respect what you don’t understand. Don’t discard it prematurely. Intuition may be a valid guide to your values.
And in firm agreement with Yudkowsky:
Love nature but don’t trust her. Be gentle once you hold enough power to protect what you love. Be open to finding new things to love.
One thought I took away was that human values are not a fixed set; they are generated by our DNA but they emerge in a complex interplay with the world. You have no “true heart”, only what you’ve built so far. So you and others might want to keep an open mind about finding new values without overwriting the old.
I hope to write more about this, but I do find technical alignment more pressing, and I still think that developers are more likely to pursue intent alignment, and to do something vaguely like a long reflection if we make it past the critical risk period. We have time for these important questions, if we have the wisdom to take it.