I hear you about not wanting to start a demon thread. At the same time, I also feel like this answer is too nonspecific for people who either want to help out or give advice. What part of your work at lightcone feels like it’s contributing to AI lab control of safety & alignment research?
At risk of seeming contrarian, I think there’s a worrying tendency within the rationalist community to look at anything broadly framed as being problematically underspecified. It’s somewhat ironic, considering that a huge focus within the field of AI safety is how dangerous it can be to build a model on bad assumptions. There’s a lot of eagerness to reduce problems down to narrowly-defined, actionable metrics that I think stems from a discomfort with the ambiguity that arises while trying to hold a lot of complexity at once.
Uncertainty seems to make rationalists disproportionately nervous, to the degree that, when confronted with concerns about Moloch/Goodhart’s Law, the response is often “But can you give us a real answer please?”, which I think is charming in an exasperating sort of way.
This is so true. I find the assumption that the required specifications are even knowable exhausting. We do not have perfect information. People be just wrong sometimes.
I hear you about not wanting to start a demon thread. At the same time, I also feel like this answer is too nonspecific for people who either want to help out or give advice. What part of your work at lightcone feels like it’s contributing to AI lab control of safety & alignment research?
At risk of seeming contrarian, I think there’s a worrying tendency within the rationalist community to look at anything broadly framed as being problematically underspecified. It’s somewhat ironic, considering that a huge focus within the field of AI safety is how dangerous it can be to build a model on bad assumptions. There’s a lot of eagerness to reduce problems down to narrowly-defined, actionable metrics that I think stems from a discomfort with the ambiguity that arises while trying to hold a lot of complexity at once.
Uncertainty seems to make rationalists disproportionately nervous, to the degree that, when confronted with concerns about Moloch/Goodhart’s Law, the response is often “But can you give us a real answer please?”, which I think is charming in an exasperating sort of way.
This is so true. I find the assumption that the required specifications are even knowable exhausting. We do not have perfect information. People be just wrong sometimes.