The option I continue to be in favor of is just give away more money with less vetting and double down on a hits based approach. Yes, we have to do some minimal vetting to make sure that the work is actually about safety and not capabilities and that the money isn’t just being grabbed by scammers, but otherwise if someone honestly wants to work on AI safety, even if it seems dumb or unlikely to work to you, just give them the money and let’s see what they produce.
I think we worry way too much about reputation concerns. These seem hypothetical to me, and if we just fund a lot of work some of it will be great and rise to the top, the rest will be mediocre and forgotten or ignored.
I’ve tried to do what I can in my own way to fund things that seem unlikely to get money from others or that have struggled to get funding, but I have limited resources, so I can only do so much. I hope others will step up and do the same.
I think we worry way too much about reputation concerns. These seem hypothetical to me, and if we just fund a lot of work some of it will be great and rise to the top, the rest will be mediocre and forgotten or ignored.
I think you’re overconfident that mediocre work will be “forgotten or ignored”. We don’t seem to have reliable metrics for measuring the goodness of alignment work. We have things like post karma and what high-status researchers are willing to say publicly about the work, but IMO these are not reliable metrics for the purpose of detecting mediocre work. (Partially due to Goodhart’s law; people who seek funding for alignment research probably tend to optimize for their posts getting high karma and their work getting endorsements from high-status researchers). FWIW I don’t think the reputation concerns are merely hypothetical at this point.
I guess, but I think of this as weighing two options:
worry about reputation and err towards false negatives in funding
don’t worry about reputation and err towards false positives in funding
I want to increase the chance that AI safety is addressed, and allowing more false positives in funding (since I assess most false positives to produce neutral outcomes relative to achieving AI safety) seems the better trade-off, so all else equal I prefer to worry less about reputation.
I think there are some arguments that reputation matters long term, but it’s not clear we have long enough for that to matter or that funding more stuff would actually hurt reputation, so lacking better arguments I remain convinced we should just fund more stuff more freely.
The option I continue to be in favor of is just give away more money with less vetting and double down on a hits based approach. Yes, we have to do some minimal vetting to make sure that the work is actually about safety and not capabilities and that the money isn’t just being grabbed by scammers, but otherwise if someone honestly wants to work on AI safety, even if it seems dumb or unlikely to work to you, just give them the money and let’s see what they produce.
I think we worry way too much about reputation concerns. These seem hypothetical to me, and if we just fund a lot of work some of it will be great and rise to the top, the rest will be mediocre and forgotten or ignored.
I’ve tried to do what I can in my own way to fund things that seem unlikely to get money from others or that have struggled to get funding, but I have limited resources, so I can only do so much. I hope others will step up and do the same.
I think you’re overconfident that mediocre work will be “forgotten or ignored”. We don’t seem to have reliable metrics for measuring the goodness of alignment work. We have things like post karma and what high-status researchers are willing to say publicly about the work, but IMO these are not reliable metrics for the purpose of detecting mediocre work. (Partially due to Goodhart’s law; people who seek funding for alignment research probably tend to optimize for their posts getting high karma and their work getting endorsements from high-status researchers). FWIW I don’t think the reputation concerns are merely hypothetical at this point.
I guess, but I think of this as weighing two options:
worry about reputation and err towards false negatives in funding
don’t worry about reputation and err towards false positives in funding
I want to increase the chance that AI safety is addressed, and allowing more false positives in funding (since I assess most false positives to produce neutral outcomes relative to achieving AI safety) seems the better trade-off, so all else equal I prefer to worry less about reputation.
I think there are some arguments that reputation matters long term, but it’s not clear we have long enough for that to matter or that funding more stuff would actually hurt reputation, so lacking better arguments I remain convinced we should just fund more stuff more freely.