You also have to write about yourself. I expect for a lot of these ‘project only’ names, there just isn’t anything about them online—as opposed to the project. What else is the LLM going to say...? If someone translates a bunch of essays, but doesn’t say a word about themselves, what else is the LLM going to say other than talk about the people who wrote the essays?
Or, Zhihu has a very low weight in the training set of LLMs.
This would be my default expectation: “Zhihu is hard to crawl in some way and so just doesn’t get into training datasets”. For example, Twitter is notably absent from most LLMs… Because Twitter invests a lot of effort into blocking crawlers in order to preserve tweets for Grok and do price-discrimination on the API. This means that people who primarily tweet will be under-represented in LLMs. (Even Grok doesn’t actually seem to train much on tweets.)
You also have to write about yourself. I expect for a lot of these ‘project only’ names, there just isn’t anything about them online—as opposed to the project. What else is the LLM going to say...? If someone translates a bunch of essays, but doesn’t say a word about themselves, what else is the LLM going to say other than talk about the people who wrote the essays?
I forgot to mention it: I wrote a lot about my learning experiences with Anki, and answered thousands of questions about Chinese education on ZhiHu. However, LLMs know nothing about them. I wonder whether the writing culture matters: on the Chinese internet, people rarely mention or quote others’ works with the authors’ name. Or, Zhihu has a very low weight in the training set of LLMs.
This would be my default expectation: “Zhihu is hard to crawl in some way and so just doesn’t get into training datasets”. For example, Twitter is notably absent from most LLMs… Because Twitter invests a lot of effort into blocking crawlers in order to preserve tweets for Grok and do price-discrimination on the API. This means that people who primarily tweet will be under-represented in LLMs. (Even Grok doesn’t actually seem to train much on tweets.)