No, it seems highly unlikely. Considered from a purely commercial perspective—which I think is the right one when considering the incentives—they are terrible customers! Consider:
That is good news! Though to be clear, I expect the default path by which they would become your customers, after some initial period of using your products or having some partnership with them, would be via acquisition, which I think avoids most of the issues that you are talking about here (in general “building an ML business with the plan of being acquired by a frontier company” has worked pretty well as a business model so far).
Whatever techniques end up being good are likely to be major modifications to training stack that would be hard to integrate, so the options for doing such a deal without revealing IP are extremely limited, making cutting us out easy.
Agree on the IP point, but I am surprised that you say that most techniques would end up major modifications to the training stack. The default product I was imagining is “RL on interpretability proxies of unintended behavior”, and I think you could do that purely in post-training. I might be wrong here, I haven’t thought that much about it, but my guess is it would just work?
I do notice I feel pretty confused about what’s going on here. Your investors clearly must have some path to profitability in mind, and it feels to me that frontier model training is really where all the money is at. Do people expect lots of smaller specialized models to be deployed? What game is there in town that isn’t frontier model training for this kind of training technique, if it does improve capabilities substantially?
You know your market better, and so I do update when you say that you don’t see your techniques used for frontier model training, but I do find myself pretty confused what the stories in the actual eyes of investors is (and you might not be able to tell me for some reason or another), and the flags I mentioned make me hesitant to update too much on your word here. So for now I will thank you for you saying otherwise, make a medium-sized positive update, and would be interested if you could expand a bit on what the actual path to profitability is without routing through frontier model training. But I understand if you don’t want to! I already appreciate your contributions here quite a bit.
That is good news! Though to be clear, I expect the default path by which they would become your customers, after some initial period of using your products or having some partnership with them, would be via acquisition, which I think avoids most of the issues that you are talking about here (in general “building an ML business with the plan of being acquired by a frontier company” has worked pretty well as a business model so far).
Agree on the IP point, but I am surprised that you say that most techniques would end up major modifications to the training stack. The default product I was imagining is “RL on interpretability proxies of unintended behavior”, and I think you could do that purely in post-training. I might be wrong here, I haven’t thought that much about it, but my guess is it would just work?
I do notice I feel pretty confused about what’s going on here. Your investors clearly must have some path to profitability in mind, and it feels to me that frontier model training is really where all the money is at. Do people expect lots of smaller specialized models to be deployed? What game is there in town that isn’t frontier model training for this kind of training technique, if it does improve capabilities substantially?
You know your market better, and so I do update when you say that you don’t see your techniques used for frontier model training, but I do find myself pretty confused what the stories in the actual eyes of investors is (and you might not be able to tell me for some reason or another), and the flags I mentioned make me hesitant to update too much on your word here. So for now I will thank you for you saying otherwise, make a medium-sized positive update, and would be interested if you could expand a bit on what the actual path to profitability is without routing through frontier model training. But I understand if you don’t want to! I already appreciate your contributions here quite a bit.