Disclaimer: I’m writing this with the awareness that Zvi has done research and synthesis, that I am probably not capable of and certainly have not been doing; and that in this specific instance, Zvi’s research has run into my area of expertise, and I can make some helpful constructive criticism in the hopes that in the near future, it will help Zvi outperform journalists even further than he already has.
Twitter spent a lot of time preparing for the “algorithm” release, at lease one month and probably many more (possibly the idea was proposed years ago). This implies that they had plenty of time to change their actual systems into something palatable for open source scrutiny.
This is exactly the sort of thing that we would see in a world where 1) a social media platform faced an existential threat, 2) distrust was one of the main factors, and 3) they still had enough talented engineers to think this up, evaluate the feasibility, and ultimately pull off a policy like this.
Whether the algorithm we see facilitates manipulation is a much more difficult question to answer. Like bills written with loopholes and encryption built with backdoors, we don’t know how easy this system is to hijack with things such as likes that were strategically placed by botnets. Establishing whether manipulation remains feasible (primarily by third party actors, which is what you should expect) is a security mindset question, thinking about how things could be (easily) broken; not a question of whether the day-to-day stuff seems like it fits together.
Regarding the bot detection, I’m not surprised that LLM bots leave artifacts behind, but I don’t think they should generally be this easy to spot in 2023. Botnets and bot detection have been trying to use AI adversarially for nearly 10 years, and probably been gainfully iterated on by engineers at large companies for ~5 years. There’s probably other obvious artifacts that a person can spot, and maybe “as an AI language model” is less of an edge case than I think it is (I don’t have much experience with language models), but it definitely seems like an edge case of bots being much easier to spot than they should be. It’s important to note that not all botnet wielders have the same level of competence; things are moving fast with GPT-4, and I wouldn’t be surprised if I vastly underestimated the number of basement people who get away with mistake-filled operations.
Disclaimer: I’m writing this with the awareness that Zvi has done research and synthesis, that I am probably not capable of and certainly have not been doing; and that in this specific instance, Zvi’s research has run into my area of expertise, and I can make some helpful constructive criticism in the hopes that in the near future, it will help Zvi outperform journalists even further than he already has.
Twitter spent a lot of time preparing for the “algorithm” release, at lease one month and probably many more (possibly the idea was proposed years ago). This implies that they had plenty of time to change their actual systems into something palatable for open source scrutiny.
This is exactly the sort of thing that we would see in a world where 1) a social media platform faced an existential threat, 2) distrust was one of the main factors, and 3) they still had enough talented engineers to think this up, evaluate the feasibility, and ultimately pull off a policy like this.
Whether the algorithm we see facilitates manipulation is a much more difficult question to answer. Like bills written with loopholes and encryption built with backdoors, we don’t know how easy this system is to hijack with things such as likes that were strategically placed by botnets. Establishing whether manipulation remains feasible (primarily by third party actors, which is what you should expect) is a security mindset question, thinking about how things could be (easily) broken; not a question of whether the day-to-day stuff seems like it fits together.
Regarding the bot detection, I’m not surprised that LLM bots leave artifacts behind, but I don’t think they should generally be this easy to spot in 2023. Botnets and bot detection have been trying to use AI adversarially for nearly 10 years, and probably been gainfully iterated on by engineers at large companies for ~5 years. There’s probably other obvious artifacts that a person can spot, and maybe “as an AI language model” is less of an edge case than I think it is (I don’t have much experience with language models), but it definitely seems like an edge case of bots being much easier to spot than they should be. It’s important to note that not all botnet wielders have the same level of competence; things are moving fast with GPT-4, and I wouldn’t be surprised if I vastly underestimated the number of basement people who get away with mistake-filled operations.