I agree, it shows the ease of shoddy copying. But it doesn’t show the ease of reverse engineering or parallel engineering.
It’s just distillation you see. It doesn’t reveal how o1 could be constructed, it just reveals how to efficiently copy from o1-like outputs (not from scratch). In other words, this recipe won’t be able to make o1, unless o1 already exists. This lets someone catch up to the leader, but not surpass them.
There are some papers that attempt to replicate o1 though, but so far they don’t quite get there. Again they are using distillation from a larger model (math-star, huggingface TTC) or not getting the same performance (see my post). Maybe we will see open source replication in a couple of months? Which means only a short lag.
It’s worth noting that Silicon Valley leaks like a sieve. And this is a feature, not a bug. Part of the reason it became the techno-VC centre of the world is because they banned non-competes. So you can deniably take your competitor’s trade secrets if you are willing to pay millions to poach some of their engineers. This is why some ML engineers get paid millions, it’s not the skill, it’s the trade secrets that competitors are paying for (and sometimes the brand-name). This has been great for tech and civilisation, but it’s not so great for maintaining a technology lead.
I agree, it shows the ease of shoddy copying. But it doesn’t show the ease of reverse engineering or parallel engineering.
It’s just distillation you see. It doesn’t reveal how o1 could be constructed, it just reveals how to efficiently copy from o1-like outputs (not from scratch). In other words, this recipe won’t be able to make o1, unless o1 already exists. This lets someone catch up to the leader, but not surpass them.
There are some papers that attempt to replicate o1 though, but so far they don’t quite get there. Again they are using distillation from a larger model (math-star, huggingface TTC) or not getting the same performance (see my post). Maybe we will see open source replication in a couple of months? Which means only a short lag.
It’s worth noting that Silicon Valley leaks like a sieve. And this is a feature, not a bug. Part of the reason it became the techno-VC centre of the world is because they banned non-competes. So you can deniably take your competitor’s trade secrets if you are willing to pay millions to poach some of their engineers. This is why some ML engineers get paid millions, it’s not the skill, it’s the trade secrets that competitors are paying for (and sometimes the brand-name). This has been great for tech and civilisation, but it’s not so great for maintaining a technology lead.