Thanks Brendon, I agree with a lot of this! I do think there’s a big open question about how capable autoGPT-like systems will end up being compared to more straightforward RL approaches. It could turn out that systems with a clear cognitive architecture just don’t work that well, even though they are safer
Yep, I agree that there’s a significant chance/risk that alternative AI approaches that aren’t as safe as LMAs are developed, and are more effective than LMAs when run in a standalone manner. I think that SCAs can still be useful in those scenarios though, definitely from a safety perspective, and less clear from a performance perspective.
For example, those models could still do itemized, sandboxed, and heavily reviewed bits of cognition inside an architecture, even though that’s not necessary for them to achieve what the architecture working towards. Also, this is when we start getting into more advanced safety features, like building symbolic/neuro-symbolic white box reasoning systems that are interpretable, for the purpose of either controlling cognition or validating the cognition of black box models (Davidad’s proposal involves the latter).
Thanks Brendon, I agree with a lot of this! I do think there’s a big open question about how capable autoGPT-like systems will end up being compared to more straightforward RL approaches. It could turn out that systems with a clear cognitive architecture just don’t work that well, even though they are safer
Yep, I agree that there’s a significant chance/risk that alternative AI approaches that aren’t as safe as LMAs are developed, and are more effective than LMAs when run in a standalone manner. I think that SCAs can still be useful in those scenarios though, definitely from a safety perspective, and less clear from a performance perspective.
For example, those models could still do itemized, sandboxed, and heavily reviewed bits of cognition inside an architecture, even though that’s not necessary for them to achieve what the architecture working towards. Also, this is when we start getting into more advanced safety features, like building symbolic/neuro-symbolic white box reasoning systems that are interpretable, for the purpose of either controlling cognition or validating the cognition of black box models (Davidad’s proposal involves the latter).