I’m curious if you (or other readers) agree with my general analysis here that among concerning + near-term superhuman abilities, hacking is near the bottom of the top 10(though still in the top 10).
I’m surprised biosafety concerns aren’t mentioned in Thane’s quick take and take on a low rank in your list. If you are strictly worried about species extinction and nothing else, then it makes sense. But if you’re more generally worried that AI capability improvements might kill you and/or your loved ones, the unlocking of new bioengineering skills around Mythos level, and their subsequent diffusion into the general public via some open weight model this year or early next year, seem like a major consideration to me. https://www.science.org/doi/10.1126/science.aej8512 is one recent example why this isn’t a purely hypothetical concern.
To be more precise, the Science article is about bacteriophages designed with Evo 2, which has been reported a while ago elsewhere, so it’s not a good example. https://www.anthropic.com/news/improving-fable-5-s-biology-safeguards is a better example of the concern that general models have reached a level where they can provide significant uplift to bad actors in the domains of molecular biology and bioengineering.
I’m curious if you (or other readers) agree with my general analysis here that among concerning + near-term superhuman abilities, hacking is near the bottom of the top 10 (though still in the top 10).
I’m surprised biosafety concerns aren’t mentioned in Thane’s quick take and take on a low rank in your list. If you are strictly worried about species extinction and nothing else, then it makes sense. But if you’re more generally worried that AI capability improvements might kill you and/or your loved ones, the unlocking of new bioengineering skills around Mythos level, and their subsequent diffusion into the general public via some open weight model this year or early next year, seem like a major consideration to me. https://www.science.org/doi/10.1126/science.aej8512 is one recent example why this isn’t a purely hypothetical concern.
To be more precise, the Science article is about bacteriophages designed with Evo 2, which has been reported a while ago elsewhere, so it’s not a good example. https://www.anthropic.com/news/improving-fable-5-s-biology-safeguards is a better example of the concern that general models have reached a level where they can provide significant uplift to bad actors in the domains of molecular biology and bioengineering.
I agree it’s a real and serious threat, just think it’s lower than some of the others.