Coding got another big leap, both for professionals and amateurs.
Claude is now clearly best. I thought for my own purposes Claude Opus was already best even after GPT-4o, but not for everyone, and it was close. Now it is not so close.
Claude’s market share has always been tiny. Will it start to rapidly expand? To what extent does the market care, when most people didn’t in the past even realize they were using GPT-3.5 instead of GPT-4? With Anthropic not doing major marketing? Presumably adaptation will be slow even if they remain on top, especially in the consumer market.
Yet with what is reportedly a big jump, we could see a lot of wrappers and apps start switching over rapidly. Developers have to be more on the ball.
How long should we expect Claude 3.5 Sonnet to remain on top?
I do not expect anyone except Google or OpenAI to pose a threat any time soon.
Was surprised by the one-shot Hanson lecture result. From it, I decided to see if Claude could identify your text, so I plugged in the above after asking “Who do you think the author behind this paragraph is?” Response:
Based on the style, content, and context of this paragraph, I believe the author is likely Zvi Mowshowitz.
Here’s my reasoning:
The writing style is analytical and forward-looking, characteristic of someone who frequently discusses AI developments and their implications.
The author mentions Claude specifically and compares it to GPT models, showing familiarity with different AI models and their capabilities.
The discussion of market share, adoption rates, and speculation about future developments is typical of someone who closely follows the AI industry.
The mention of “wrappers and apps” and consideration of developer behavior suggests someone with knowledge of the AI ecosystem.
Zvi Mowshowitz is known for writing about AI developments, particularly on his blog “Don’t Worry About the Vase” and other platforms. He often provides analysis of new AI models and their potential impacts.
The structure and tone of the writing is similar to Zvi’s typical style, with a mix of personal opinion and broader industry analysis.
However, I should note that while this is my best guess based on the available information, I can’t be absolutely certain. If you’re looking for a definitive answer, it would be best to verify the source directly.
Would you like me to elaborate on any part of this analysis?
As I mentioned on Twitter, this sort of ‘truesight’ for writers extensively represented in Internet corpora like Robin Hanson, Zvi, or myself, is very unsurprising. Like those slides—there are not a lot of places other than Overcoming Bias in the 2000s that all of those topics are represented. (Hanson has been banging those drums for a long time.)
I gave it a few paragraphs from something I posted on Mastodon yesterday, and it identified me. I’m at least a couple of notches less internet-famous than Zvi or gwern, though again there’s a fair bit of my writing on the internet and my style is fairly distinctive. I’m quite impressed.
(I then tried an obvious thing and fed it a couple of Bitcoin-white-paper paragraphs, but of course it knew that they were “Satoshi Nakamoto” and wasn’t able to get past that. Someone sufficiently determined to identify Satoshi and with absurd resources could do worse than to train a big LLM on “everything except writings explicitly attributed to Satoshi Nakamoto” and then see what it thinks.)
Was surprised by the one-shot Hanson lecture result. From it, I decided to see if Claude could identify your text, so I plugged in the above after asking “Who do you think the author behind this paragraph is?” Response:
Wow
As I mentioned on Twitter, this sort of ‘truesight’ for writers extensively represented in Internet corpora like Robin Hanson, Zvi, or myself, is very unsurprising. Like those slides—there are not a lot of places other than Overcoming Bias in the 2000s that all of those topics are represented. (Hanson has been banging those drums for a long time.)
I gave it a few paragraphs from something I posted on Mastodon yesterday, and it identified me. I’m at least a couple of notches less internet-famous than Zvi or gwern, though again there’s a fair bit of my writing on the internet and my style is fairly distinctive. I’m quite impressed.
(I then tried an obvious thing and fed it a couple of Bitcoin-white-paper paragraphs, but of course it knew that they were “Satoshi Nakamoto” and wasn’t able to get past that. Someone sufficiently determined to identify Satoshi and with absurd resources could do worse than to train a big LLM on “everything except writings explicitly attributed to Satoshi Nakamoto” and then see what it thinks.)
For Satoshi scenarios where you have a very small corpus or the corpus is otherwise problematic (in this case, you can’t easily get new Satoshi text heldout from training), you could do things like similarity/distance metrics: https://www.lesswrong.com/posts/dLg7CyeTE4pqbbcnp/language-models-model-us?commentId=MNk22rZeELjoh7bhW