I can’t quite tell why you think Twitch is bad. It is subject to network effects, kind of a social media company, is that why? And I don’t know what Scale.com is other than some AI company.
Scale’s mission is something like accelerating AI progress, and they have no safety department. So ¯\_(ツ)_/¯ For Twitch I think a bunch of good stuff happens there (chess streamers, Ed Kmett streaming Haskell, or just great gamers), but they’re also in a domain where clickbait and similar Goodharting dynamics are strong, and in the worlds where it gets really big I expect those to dominate.
On the topic of tails, I wonder if your distribution would’ve come out differently had the scale been −10, −1, 0, 1, 10.
I think I would rarely have assigned 10s, due to it being a complex question and this just being a very rough draft.
Another interesting question is whether weighing the rankings by market cap would have made a difference. (But YC didn’t make valuations available in their data, so it would require ~30 min of data entry.)
Scale’s mission is something like accelerating AI progress, and they have no safety department. So ¯\_(ツ)_/¯ For Twitch I think a bunch of good stuff happens there (chess streamers, Ed Kmett streaming Haskell, or just great gamers), but they’re also in a domain where clickbait and similar Goodharting dynamics are strong, and in the worlds where it gets really big I expect those to dominate.
I think I would rarely have assigned 10s, due to it being a complex question and this just being a very rough draft.
Another interesting question is whether weighing the rankings by market cap would have made a difference. (But YC didn’t make valuations available in their data, so it would require ~30 min of data entry.)