If these are close enough, then AGI safety products are a reasonable idea. If not, they’re an actively bad idea because the new incentives pull you in a less impactful direction.
I’ve heard this argument a lot, but I think it has a key failure mode: it assumes incentives are more or less invariant with scale (of org size, staff, cash on hand, etc). As they say in the Valley “most problems are solved with scale”.
Another heuristic common in the startup world but which seems to not occur to people on this forum too much: many successful startups do not start intending to build in their actually successful direction—they pivot at least once.
Together, these facts to me imply a dynamic where building a successful company is both very compatible with AGI safety in the long run, but going about it by trying to backchain from AGI safety seems unlikely to work. Instead, good founders are likely to pivot around a bunch to find PMF. But even if the idea with PMF is not the maximally AGI safe one to build, that doesn’t mean it’s a bad idea! If you build that, then come back a year later with much more capital and a large team to build a second product, then you are in a significantly better position than if you had tried to build for AGI safety from the start.
I think a common failure mode amongst the AI safety startup market is “trying to build the perfect product from the start”; trying to find the narrow path that has both safety and 0 to 1 profitability for a company with no revenue or existing customers.
I’m curious for your reasons for choosing PBC. Having looked into it a little, it seems like:
There are plenty of historical records of normal for-profits doing things which nominally seem to be hurting profits but which get justified by the CEO because “it’s better for us long-term” effectively. Ben & Jerry’s is a well known example. So if you go the normal for-profit route it seems like you have freedom of action to do this. If the concern is the board, you can choose your board members wisely and/or set up an additional “advisory board” with various powers under a normal for-profit too.
PBCs are materially worse for investors, as I understand it, even if the difference is not large, and so you slightly hurt your ability to raise.