A viral article from a Chinese AI professional
That was a really interesting article. (I think it blew up on twitter a while ago.)
I am pretty sympathetic to the author. If labor income diminishes, income redistribution / UBI is necessary; it is easier to trust your home country to follow its own ideology than to trust a foreign company to share the spoils. Especially when Dario makes so many China-hawkish public statements.
In my limited experience (quantum chemistry) mature academic fields take up external ideas slowly without a full-time faculty member somewhere putting out a stream of papers, even if those ideas are clear and useful. On the bright side, if Scott Aaronson discusses it in his new course, maybe someone in the UT Austin philosophy department will get inspired..
I think this is quite interesting, and also hilarious. I once showed the FDT paper to a buddy taking an ethics class, and he laughed at it. The strong effect shown here reflects the fact that FDT has not diffused into mainstream philosophy, and is largely dismissed by practitioners who do know about it—too sci-fi for them. But now that real life is yesteryear’s sci-fi, and FDT is getting free marketing, it may grow in popularity.
It’s surprising that the models are aware of FDT to this extent. I think it’s a case of https://xkcd.com/2635/; the labs favor FDT for cultural reasons, and the models’ opinions reflect theirs.
If I were involved in academic philosophy, I too would be reluctant to discuss FDT with colleagues, because they’d probably say it’s fringe BS.
Thanks. I made some quick changes but it’s probably not structured enough for your taste. Ah well
Nice point. I think most of the time this doesn’t apply, because agents don’t just try to maximize “utility” as some abstract quantity, but possess (transparently or opaquely) some function which they concretely try to increase. Utility is a device, specific to each agent, that refers to states of the world, and isn’t a kind of universal brownie point that everyone likes.
On the other hand, putting “utility” as your utility is fertile ground for wireheading temptations. I define my utility to be ! There, I win.
Thanks for posting the followup to LSY’s original article, which I hadn’t seen before. He writes in a really sincere way. I do hope everyone going off from LW to discuss AI safety on the Chinese internet has already engaged with it.