It might cost multiple orders of magnitude more than $10B to build human-level AI. I could still see a similar scenario playing out if the baseline cost is $100B, but probably not at $1T. As I understand, present-day models cost more like $100B (edit: I badly misread a graph, present-day models cost more like $100M although the figures are not public); the first human-level AI will probably cost considerably more.
I doubt a 100x increase in spending (or 1000x) is enough to go from human-level to superintelligent, but I don’t think we can strongly rule it out. We don’t know if scaling laws will continue to hold, and also we don’t know what level of intelligence is required for an AI to pose an existential threat. (Like, maybe 150 IQ + the ability to clone yourself is already sufficient. Probably not, but maybe.)
I somewhat doubt AI companies would decide to do this. It contradicts their stated plans, and it would be a deviation from their historical behavior. But once AI gets good enough to replace human workers, its profitability rapidly increases, so it could be economically justifiable to do a fast scale-up even though that wasn’t justified at weaker capability levels.
Answering my own question:
It might cost multiple orders of magnitude more than $10B to build human-level AI. I could still see a similar scenario playing out if the baseline cost is $100B, but probably not at $1T. As I understand,
present-day models cost more like $100B(edit: I badly misread a graph, present-day models cost more like $100M although the figures are not public); the first human-level AI will probably cost considerably more.I doubt a 100x increase in spending (or 1000x) is enough to go from human-level to superintelligent, but I don’t think we can strongly rule it out. We don’t know if scaling laws will continue to hold, and also we don’t know what level of intelligence is required for an AI to pose an existential threat. (Like, maybe 150 IQ + the ability to clone yourself is already sufficient. Probably not, but maybe.)
I somewhat doubt AI companies would decide to do this. It contradicts their stated plans, and it would be a deviation from their historical behavior. But once AI gets good enough to replace human workers, its profitability rapidly increases, so it could be economically justifiable to do a fast scale-up even though that wasn’t justified at weaker capability levels.