I like this comment. I think it’s easy to overfit on the most salient research agendas, especially if there are echo chambers and tight coupling between highly paid frontier AI staff and nonprofit funders. The best way I know to combat this at MATS is:
Maintain a broad church of AI safety research, including deliberately making mentor “diversity picks” and choosing a mentor selection committee that contains divergent thinkers. As another example, I think Constellation has done a good job recently at expanding member diversity and reducing echo chambers.
Requiring that COIs be declared and mitigated, including along reporting chains, at the same organization, with romantic/sexual partners, and with frequent research collaborators.
Encouraging “scout mindset” and “reasoning transparency”, especially among people with divergent beliefs. I think this is a large strength of MATS: we are a melting pot for ideas and biases.
Note that I expect overfitting to decrease with further scale and diversity, given the above practices are adhered to!
I like this comment. I think it’s easy to overfit on the most salient research agendas, especially if there are echo chambers and tight coupling between highly paid frontier AI staff and nonprofit funders. The best way I know to combat this at MATS is:
Maintain a broad church of AI safety research, including deliberately making mentor “diversity picks” and choosing a mentor selection committee that contains divergent thinkers. As another example, I think Constellation has done a good job recently at expanding member diversity and reducing echo chambers.
Requiring that COIs be declared and mitigated, including along reporting chains, at the same organization, with romantic/sexual partners, and with frequent research collaborators.
Encouraging “scout mindset” and “reasoning transparency”, especially among people with divergent beliefs. I think this is a large strength of MATS: we are a melting pot for ideas and biases.
Note that I expect overfitting to decrease with further scale and diversity, given the above practices are adhered to!