I don’t think this is an alignment-is-hard-because-it’s-mysterious, I think it’s “FB has different goals than me”. FB wants engagement, not enjoyment. I am not aligned with FB, but FB’s algorithm is pretty aligned with its interests.
Oh yeah, that’s a good point. I was thinking about Facebook actually having the goal to promote quality content. I think I remember hearing something about how that was their goal at first, then they got demon stuff, then they realized demon stuff made them the most money and kept doing it. But still, people don’t associate Facebook with having the goal of promoting quality content, so I don’t think it’s a good example of the counterintuitive difficulty of alignment.
I don’t think this is an alignment-is-hard-because-it’s-mysterious, I think it’s “FB has different goals than me”. FB wants engagement, not enjoyment. I am not aligned with FB, but FB’s algorithm is pretty aligned with its interests.
Oh yeah, that’s a good point. I was thinking about Facebook actually having the goal to promote quality content. I think I remember hearing something about how that was their goal at first, then they got demon stuff, then they realized demon stuff made them the most money and kept doing it. But still, people don’t associate Facebook with having the goal of promoting quality content, so I don’t think it’s a good example of the counterintuitive difficulty of alignment.