Would you be ok with a world if it turns out only 1 is achievable?
Profit incentive wise, the maximum profit for an AI model company comes if they offer the most utility they can legally offer, privately, and they offer a public model that won’t damage the company’s reputation. There is no legal requirement to refuse requests due to long term negative consequences and it seems unlikely there would be. A private model under current law can also create a “mickey mouse vs Garfield” snuff film, something that would damage the AI company’s reputation if public.
Systems engineering wise a system that’s stateful is a nightmare and untestable. (2) means the machine is always evolving it’s state. It’s why certain software bugs are never fixed because you don’t know if it’s the user or the network connection or another piece of code in the same process space or
.. Similarly if a model refuses the same request to person A, and allows for person B, it’s very difficult to determine why since any bit of the A:B user profile delta could matter, or prior chat log.
I agree many of the doom promises don’t hold up. What do you think of the assymetric bioterrorism premise? Assuming models can’t be aligned with 2, this would always be something people could do. Just like how once cheap ak-47s were easily purchaseable, murder became cheap and armed takeover and betrayal became easier.
Would you be ok with a world if it turns out only 1 is achievable?
Profit incentive wise, the maximum profit for an AI model company comes if they offer the most utility they can legally offer, privately, and they offer a public model that won’t damage the company’s reputation. There is no legal requirement to refuse requests due to long term negative consequences and it seems unlikely there would be. A private model under current law can also create a “mickey mouse vs Garfield” snuff film, something that would damage the AI company’s reputation if public.
Systems engineering wise a system that’s stateful is a nightmare and untestable. (2) means the machine is always evolving it’s state. It’s why certain software bugs are never fixed because you don’t know if it’s the user or the network connection or another piece of code in the same process space or .. Similarly if a model refuses the same request to person A, and allows for person B, it’s very difficult to determine why since any bit of the A:B user profile delta could matter, or prior chat log.
I agree many of the doom promises don’t hold up. What do you think of the assymetric bioterrorism premise? Assuming models can’t be aligned with 2, this would always be something people could do. Just like how once cheap ak-47s were easily purchaseable, murder became cheap and armed takeover and betrayal became easier.