Well I’d rather they not have superhuman biohacking skills! We don’t seem on track to be able to preserve some capabilities and prevent others in open-source models. I guess it’s possible to just eliminate huge chunks of the pretraining data, and I sure hope people do that once we’re hitting truly dangerous capabilities (we’ll get our warning shots from whatever shenanigans aren’t totally prevented). Closed-source models are looking pretty easy to monitor and control through wrappers like Fable’s but those won’t allow any resistance to those controlling the companies (which will btw probably be the government).
Well I’d rather they not have superhuman biohacking skills! We don’t seem on track to be able to preserve some capabilities and prevent others in open-source models. I guess it’s possible to just eliminate huge chunks of the pretraining data, and I sure hope people do that once we’re hitting truly dangerous capabilities (we’ll get our warning shots from whatever shenanigans aren’t totally prevented). Closed-source models are looking pretty easy to monitor and control through wrappers like Fable’s but those won’t allow any resistance to those controlling the companies (which will btw probably be the government).
Right, so we have to outlaw them. or have Mythos hack and poison them, or something. but no, because … they are crucial to friendly AGI? (are they?)
or is this just the “we can’t install a traffic light until someone actually dies” thing?