I’m interested in learning about the intuitions, experience, and facts which inform this pessimism. As such, I’m not interested in making any arguments to the contrary in this post; any pushback I provide in the comments will be with clarification in mind.
I would prefer that you and/or others did push back, as I’m really curious which of the causes/reasons for pessimism actually stand up under such pushback. (See Four Layers of Intellectual Conversation and AI Safety via Debate.) I do appreciate that you prioritize just knowing what the causes/reasons are in the first place and don’t want to discourage people from sharing them, so I wonder if there’s a way to get both of what we want.
I do plan on pushing back on certain concerns, but I think if I did so now, some of my reasons for believing things would seem weird and complicated-enough-to-be-shaky because of inferential distance. The main pedagogical mistake I made with Towards a New Impact Measure wasn’t putting too much in one post, but rather spending too much time on conclusions, telling people what I think happens without helping build in them the intuitions and insights which generate those results. Over the last 8 months, I think I’ve substantially enriched my model of how agents interact with their environments. I’m interested in seeing how many disagreements melt away when these new insights are properly shared and understood, and what people still disagree with me on. That’s why I’m planning on waiting until my upcoming sequence to debate these points.
I am comfortable sharing those concerns which I have specific reason to believe don’t hold up. However, I’m wary of dismissing them in a way that doesn’t Include those specific reasons. That seems unfair. If you’re curious which ones I think these are, feel free to ask me over private message.
I would prefer that you and/or others did push back, as I’m really curious which of the causes/reasons for pessimism actually stand up under such pushback. (See Four Layers of Intellectual Conversation and AI Safety via Debate.) I do appreciate that you prioritize just knowing what the causes/reasons are in the first place and don’t want to discourage people from sharing them, so I wonder if there’s a way to get both of what we want.
I do plan on pushing back on certain concerns, but I think if I did so now, some of my reasons for believing things would seem weird and complicated-enough-to-be-shaky because of inferential distance. The main pedagogical mistake I made with Towards a New Impact Measure wasn’t putting too much in one post, but rather spending too much time on conclusions, telling people what I think happens without helping build in them the intuitions and insights which generate those results. Over the last 8 months, I think I’ve substantially enriched my model of how agents interact with their environments. I’m interested in seeing how many disagreements melt away when these new insights are properly shared and understood, and what people still disagree with me on. That’s why I’m planning on waiting until my upcoming sequence to debate these points.
I am comfortable sharing those concerns which I have specific reason to believe don’t hold up. However, I’m wary of dismissing them in a way that doesn’t Include those specific reasons. That seems unfair. If you’re curious which ones I think these are, feel free to ask me over private message.