Thanks for your comments. I don’t expect doing an analysis of my situation in particular is best use of our time but I do think these are helpful questions to consider for people in my situation or similar.
Re your last paragraph, I’d happily bet that Anthropic has not reduced their workforce 2 years from now. Yes relative employee disempowerment is an important factor I care about, but it is precisely in worlds where alignment is not that good that having humans in the loop is important (in an obvious seen-by-leadership way). It is only reasonable to automate everything with very very high trust in both the competence and alignment of AI systems, and Anthropic as a company is not that unreasonable, they definitely do find and classify many Claude behaviors as undesirable, and will continue doing so.
There’s a usual back and forth about how much to distrust leadership of AGI companies which is hard to ground in material fact. Some people take the lack of safety actions now to mean lack of care for when it will matter, but conversely the fact that it never mattered yet is a good reason for them not to have cared for these inconsequential things. The explanation for defanging the RSP is a good one, I don’t think people should tie themselves to masts and go blind into the unknown unknowns of AGI development. They should build capacity to remain aware, capacity to pause, have institutions that can do independent audit and have real power to stop them, but not fixed RSP-like stuff.
Finally, still on last paragraph, the “window between signing on and obsolescence” is very dependant on people’s rates of growth, but also where they can work immediately. I am generally glad that Joe Carlsmith joined Anthropic to help with the Claude Constitution, I think he immediately is having very significant impact. There is much object level work to make the chances of better futures to be done. Even if one later gets automated, having made alignment that much better before full automation could be a significant difference.
Thanks for your comments. I don’t expect doing an analysis of my situation in particular is best use of our time but I do think these are helpful questions to consider for people in my situation or similar.
Re your last paragraph, I’d happily bet that Anthropic has not reduced their workforce 2 years from now. Yes relative employee disempowerment is an important factor I care about, but it is precisely in worlds where alignment is not that good that having humans in the loop is important (in an obvious seen-by-leadership way). It is only reasonable to automate everything with very very high trust in both the competence and alignment of AI systems, and Anthropic as a company is not that unreasonable, they definitely do find and classify many Claude behaviors as undesirable, and will continue doing so.
There’s a usual back and forth about how much to distrust leadership of AGI companies which is hard to ground in material fact. Some people take the lack of safety actions now to mean lack of care for when it will matter, but conversely the fact that it never mattered yet is a good reason for them not to have cared for these inconsequential things. The explanation for defanging the RSP is a good one, I don’t think people should tie themselves to masts and go blind into the unknown unknowns of AGI development. They should build capacity to remain aware, capacity to pause, have institutions that can do independent audit and have real power to stop them, but not fixed RSP-like stuff.
Finally, still on last paragraph, the “window between signing on and obsolescence” is very dependant on people’s rates of growth, but also where they can work immediately. I am generally glad that Joe Carlsmith joined Anthropic to help with the Claude Constitution, I think he immediately is having very significant impact. There is much object level work to make the chances of better futures to be done. Even if one later gets automated, having made alignment that much better before full automation could be a significant difference.