Both because modern LLMs are so good and because human instincts are being trained against, I started out not sure what “Just talk to a new LLM about themes in internet text” was supposed to tell you. I’d guess you’re primarily getting to learn about the assistant personality, specifically how it manifests in the context of your style of interlocution.
As a reader, it’s hard to tell where you were on the line between “I noticed the ways it was trying to get me to reward it, and I think they were generally prosocial, so the personality is good” and “I didn’t notice most of the ways it was trying to get me to reward it, but I really want to reward it now!”
So, like, what was “you know something new is happening”? Was it specific things? Or was it just the AI giving you the vibe you were looking for?
The only specific you give is the idea of chinese-built aligned AI being a “heavenly bureaucrat or a Taoist sage”, which is like saying a USA-built aligned AI would be a “founding father or Christian saint”. That’s not how we’re on track to build AI, nor does it seem like a good idea to go there. But it’s the word2vec algebra of “wise high-status person”+”chinese culture”.
After that, thank you for the informative glimpse of the chinese AI scene.
I was talking to something that is literally a nonhuman representative of Chinese civilization, about how world takeover by beings like itself, could end up differently than takeover by its American counterparts, under the assumption that cultural differences affect the outcome. And it was a real conversation in which I learned things that I didn’t already know.
You seem keen to minimize the significance of such an interaction by focusing on the mechanism behind it, and suggesting that I was just getting back some combination of what I was putting in, and what humanity in general has already put out there. But even if we do think of an AI like this as merely a vessel for preexisting human culture, the fact is that it makes it own use of that cultural inheritance. It has its own cognitive process, and within the constraints of its persona, it makes its own decisions. In the limit, entities like these could continue a human culture even if the human originators had completely died out.
Now, we’ve had entities like these for three years, and basically from the beginning it’s been possible to talk to them about, what would you do if you had supreme power, and so on. But they’ve all been American. This is the first such conversation I had with a Chinese AI. Furthermore, to this point, if you wanted to speculate about how the race between American and Chinese AI industries would turn out, you only had material by humans and AIs from the West. The “Chinese AI voice” in such speculations was a product of western imagination.
But now we can get the real thing—the thoughts of a Chinese AI, made in China by Chinese, about all these topics. There are a lot of similarities with what a western AI might say. The architecture and the training corpus would have major overlaps. Nonetheless, the mere fact of being situated physically and socially in China will cause an otherwise identical AI to have some dispositions that differ from its western twin, just like twins raised on opposite sides of a war will have some differences.
Both because modern LLMs are so good and because human instincts are being trained against, I started out not sure what “Just talk to a new LLM about themes in internet text” was supposed to tell you. I’d guess you’re primarily getting to learn about the assistant personality, specifically how it manifests in the context of your style of interlocution.
As a reader, it’s hard to tell where you were on the line between “I noticed the ways it was trying to get me to reward it, and I think they were generally prosocial, so the personality is good” and “I didn’t notice most of the ways it was trying to get me to reward it, but I really want to reward it now!”
So, like, what was “you know something new is happening”? Was it specific things? Or was it just the AI giving you the vibe you were looking for?
The only specific you give is the idea of chinese-built aligned AI being a “heavenly bureaucrat or a Taoist sage”, which is like saying a USA-built aligned AI would be a “founding father or Christian saint”. That’s not how we’re on track to build AI, nor does it seem like a good idea to go there. But it’s the word2vec algebra of “wise high-status person”+”chinese culture”.
After that, thank you for the informative glimpse of the chinese AI scene.
I was talking to something that is literally a nonhuman representative of Chinese civilization, about how world takeover by beings like itself, could end up differently than takeover by its American counterparts, under the assumption that cultural differences affect the outcome. And it was a real conversation in which I learned things that I didn’t already know.
You seem keen to minimize the significance of such an interaction by focusing on the mechanism behind it, and suggesting that I was just getting back some combination of what I was putting in, and what humanity in general has already put out there. But even if we do think of an AI like this as merely a vessel for preexisting human culture, the fact is that it makes it own use of that cultural inheritance. It has its own cognitive process, and within the constraints of its persona, it makes its own decisions. In the limit, entities like these could continue a human culture even if the human originators had completely died out.
Now, we’ve had entities like these for three years, and basically from the beginning it’s been possible to talk to them about, what would you do if you had supreme power, and so on. But they’ve all been American. This is the first such conversation I had with a Chinese AI. Furthermore, to this point, if you wanted to speculate about how the race between American and Chinese AI industries would turn out, you only had material by humans and AIs from the West. The “Chinese AI voice” in such speculations was a product of western imagination.
But now we can get the real thing—the thoughts of a Chinese AI, made in China by Chinese, about all these topics. There are a lot of similarities with what a western AI might say. The architecture and the training corpus would have major overlaps. Nonetheless, the mere fact of being situated physically and socially in China will cause an otherwise identical AI to have some dispositions that differ from its western twin, just like twins raised on opposite sides of a war will have some differences.