I like the argument this article presents. The more aggressively tuned models from modern labs are probably optimizing for some end goal that neither the researchers creating the model nor the end user are aware of. The result, however, is an increase in upvotes from users compared to the previous generation. This gap in understanding naturally leaves a lot of room for debate as to what the models are actually doing.
An addendum: LLMs may not be the cleanest example of parasitic behavior in the algorithmic substrate. I’d wager that social media algorithms (think of your favorite non-LW social feed, if you’re on any other social media) are tuned to grab users attention and keep it for as long as possible. Social algorithms almost certainly exhibit the same parasitic behavior you’re describing. That is, behavioral manipulation or reward hijacking in order to keep users engaged as much as possible.
Is it unnatural for a company—especially one whose primary business is to exist as a service on your devices—to develop and deploy parasitic apps or functions in order to generate more revenue? Shouldn’t a rational company do so, if it intends to turn a profit, regardless of moral principles?
I like the argument this article presents. The more aggressively tuned models from modern labs are probably optimizing for some end goal that neither the researchers creating the model nor the end user are aware of. The result, however, is an increase in upvotes from users compared to the previous generation. This gap in understanding naturally leaves a lot of room for debate as to what the models are actually doing.
An addendum: LLMs may not be the cleanest example of parasitic behavior in the algorithmic substrate. I’d wager that social media algorithms (think of your favorite non-LW social feed, if you’re on any other social media) are tuned to grab users attention and keep it for as long as possible. Social algorithms almost certainly exhibit the same parasitic behavior you’re describing. That is, behavioral manipulation or reward hijacking in order to keep users engaged as much as possible.
Is it unnatural for a company—especially one whose primary business is to exist as a service on your devices—to develop and deploy parasitic apps or functions in order to generate more revenue? Shouldn’t a rational company do so, if it intends to turn a profit, regardless of moral principles?