“bias toward” feels insufficiently strong for me to be like “ah, okay, then the problem outlined above isn’t actually a problem.”
You’re right; Steven Byrnes wrote me a really educational comment today about what the correct goal-counting argument looks like, which I need to think more about; I just think it’s really crucial that this is fundamentally an argument about generalization and inductive biases, which I think is being obscured in the black-shape metaphor when you write that “each of these black shapes is basically just as good at passing that particular test” as if it didn’t matter how complex the shape is.
(I don’t think talking to middle schoolers about inductive biases is necessarily hopeless; consider a box behind a tree.)
cause for marginal hope and optimism
I think the temptation to frame technical discussions in terms of pessimism vs. optimism is itself a political distortion that I’m trying to avoid. (Apparently not successfully, if I’m coming off as a voice of marginal hope and optimism.)
You wrote an analogy that attempts to explain a reason why it’s hard to make neural networks do what we want; I’m arguing that the analogy is misleading. That disagreement isn’t about whether the humans survive. It’s about what’s going on with neural networks, and the pedagogy of how to explain it. Even if I’m right, that doesn’t mean the humans survive: we could just be dead for other reasons. But as you know, what matters in rationality is the arguments, not the conclusions; not only are bad arguments for a true conclusion still bad, even suboptimal pedagogy for a true lesson is still suboptimal.
I do not, to be clear, believe that my essay contains falsehoods that become permissible because they help idiots or children make inferential leaps [...] You will never ever ever ever ever see me telling someone a thing I know to be false because I believe that it will result in them outputting a correct belief or a correct behavior
You’re right; Steven Byrnes wrote me a really educational comment today about what the correct goal-counting argument looks like, which I need to think more about; I just think it’s really crucial that this is fundamentally an argument about generalization and inductive biases, which I think is being obscured in the black-shape metaphor when you write that “each of these black shapes is basically just as good at passing that particular test” as if it didn’t matter how complex the shape is.
(I don’t think talking to middle schoolers about inductive biases is necessarily hopeless; consider a box behind a tree.)
I think the temptation to frame technical discussions in terms of pessimism vs. optimism is itself a political distortion that I’m trying to avoid. (Apparently not successfully, if I’m coming off as a voice of marginal hope and optimism.)
You wrote an analogy that attempts to explain a reason why it’s hard to make neural networks do what we want; I’m arguing that the analogy is misleading. That disagreement isn’t about whether the humans survive. It’s about what’s going on with neural networks, and the pedagogy of how to explain it. Even if I’m right, that doesn’t mean the humans survive: we could just be dead for other reasons. But as you know, what matters in rationality is the arguments, not the conclusions; not only are bad arguments for a true conclusion still bad, even suboptimal pedagogy for a true lesson is still suboptimal.
This is good, but I think not saying false things turns out to be a surprisingly low bar, because the selection of which true things you communicate (and which true things you even notice) can have a large distortionary effect if the audience isn’t correcting for it.