I think I object level disagree with you on the china vector of existential risk, I think it is a self-fulfilling prophecy and that it does not engage with the current AI situation in china.
If you were object-level correct about china I would agree with the post but I just think you’re plain wrong.
Do you disagree that entities in China are now pushing the state of the art in an open source way?
If you disagree, then sure, you don’t have to update. But I’d argue you aren’t paying attention.
If you agree, then how did you update?
If your point is that using ‘use vs them’ framing makes thing worse, that may or may not be correct, but from the perspective of existential risk the object level determination re China is irrelevant, vs what “they” represent. A repeated game where defection by anyone one of N players leads to ruin (from the doomer perspective) and where folks in China just represent one of a very large set.
So I guess the point then more becomes about general open source development of other countries where China is part of it and that people did not correctly predict this as something that would happen.
Something like distillation techniques for LLMs would be used by other countries and then profilerated and that the rationality community as a whole did not take this into account?
I’ll agree with you that Bayes points should be lost in prediction of theory of mind of nation states, it is quite clear that they would be interested in this from a macro-analysis perspective (I say in hindsight of course.)
I’m not sure that Deepseek is SOTA in terms of inherent development, it seems to me that they’re using some of the existing work from OpenAI, Deepmind & Anthropic but I might be wrong here, is there anything else that you’re pointing at?
There are many math and coding benchmarks where models from DeepSeek, Ali baba and tencent are now leading, and definitely leading what was SOTA a year ago. If you don’t want to take my word for it I can dig them up.
No, we good. I was just operating under the assumption that deepseek was just doing distilling of OpenAI but it doesnt seem to be the only good ML company from China. There’s also a bunch of really good ML researchers from China so I agree at this point.
I think I object level disagree with you on the china vector of existential risk, I think it is a self-fulfilling prophecy and that it does not engage with the current AI situation in china.
If you were object-level correct about china I would agree with the post but I just think you’re plain wrong.
Here’s a link to a post that makes some points about the general epistmic situation around china: https://www.lesswrong.com/posts/uRyKkyYstxZkCNcoP/careless-talk-on-us-china-ai-competition-and-criticism-of
Do you disagree that entities in China are now pushing the state of the art in an open source way?
If you disagree, then sure, you don’t have to update. But I’d argue you aren’t paying attention.
If you agree, then how did you update?
If your point is that using ‘use vs them’ framing makes thing worse, that may or may not be correct, but from the perspective of existential risk the object level determination re China is irrelevant, vs what “they” represent. A repeated game where defection by anyone one of N players leads to ruin (from the doomer perspective) and where folks in China just represent one of a very large set.
Does that make sense?
So I guess the point then more becomes about general open source development of other countries where China is part of it and that people did not correctly predict this as something that would happen.
Something like distillation techniques for LLMs would be used by other countries and then profilerated and that the rationality community as a whole did not take this into account?
I’ll agree with you that Bayes points should be lost in prediction of theory of mind of nation states, it is quite clear that they would be interested in this from a macro-analysis perspective (I say in hindsight of course.)
I’m not sure that Deepseek is SOTA in terms of inherent development, it seems to me that they’re using some of the existing work from OpenAI, Deepmind & Anthropic but I might be wrong here, is there anything else that you’re pointing at?
There are many math and coding benchmarks where models from DeepSeek, Ali baba and tencent are now leading, and definitely leading what was SOTA a year ago. If you don’t want to take my word for it I can dig them up.
No, we good. I was just operating under the assumption that deepseek was just doing distilling of OpenAI but it doesnt seem to be the only good ML company from China. There’s also a bunch of really good ML researchers from China so I agree at this point.