At risk of seeming contrarian, I think there’s a worrying tendency within the rationalist community to look at anything broadly framed as being problematically underspecified. It’s somewhat ironic, considering that a huge focus within the field of AI safety is how dangerous it can be to build a model on bad assumptions. There’s a lot of eagerness to reduce problems down to narrowly-defined, actionable metrics that I think stems from a discomfort with the ambiguity that arises while trying to hold a lot of complexity at once.
Uncertainty seems to make rationalists disproportionately nervous, to the degree that, when confronted with concerns about Moloch/Goodhart’s Law, the response is often “But can you give us a real answer please?”, which I think is charming in an exasperating sort of way.
This is so true. I find the assumption that the required specifications are even knowable exhausting. We do not have perfect information. People be just wrong sometimes.
At risk of seeming contrarian, I think there’s a worrying tendency within the rationalist community to look at anything broadly framed as being problematically underspecified. It’s somewhat ironic, considering that a huge focus within the field of AI safety is how dangerous it can be to build a model on bad assumptions. There’s a lot of eagerness to reduce problems down to narrowly-defined, actionable metrics that I think stems from a discomfort with the ambiguity that arises while trying to hold a lot of complexity at once.
Uncertainty seems to make rationalists disproportionately nervous, to the degree that, when confronted with concerns about Moloch/Goodhart’s Law, the response is often “But can you give us a real answer please?”, which I think is charming in an exasperating sort of way.
This is so true. I find the assumption that the required specifications are even knowable exhausting. We do not have perfect information. People be just wrong sometimes.