I want to flag this for the future, but I suspect a huge underlying crux might be whether something like a persistent memory/identity and reliability to go with is just fundamentally necessary for many tasks to be done, with realistic compute and in practice you can’t take shortcuts to unlock most of the value of AI.
This is related to whether you can reduce the long-term memory of humans into a huge context window, and I currently am not highly confident that the answer is no (that would require more scale-up and more time), though I do slightly favor the hypothesis that the answer is no over the answer is yes.
Putting it another way, the crux is whether can get most tasks to be automated by short-lifetime AIs, rather than long, individual lifetime AIs, and I’m not nearly as confident as you or @1a3orn that long-term memory isn’t necessary and not having it pays a crippling capability tax.
I want to flag this for the future, but I suspect a huge underlying crux might be whether something like a persistent memory/identity and reliability to go with is just fundamentally necessary for many tasks to be done, with realistic compute and in practice you can’t take shortcuts to unlock most of the value of AI.
This is related to whether you can reduce the long-term memory of humans into a huge context window, and I currently am not highly confident that the answer is no (that would require more scale-up and more time), though I do slightly favor the hypothesis that the answer is no over the answer is yes.
Link below:
https://www.lesswrong.com/posts/oKAFFvaouKKEhbBPm/a-bear-case-my-predictions-regarding-ai-progress#9LT2RgZiqgTQHudpm
This is also a crux I suspect for @1a3orn, based on this response to AI 2027 (though it focuses on different criticisms):
https://www.lesswrong.com/posts/4MofJAmXDkoFsTf4B/?commentId=EJZFG8m6ETWgZRFJi
Putting it another way, the crux is whether can get most tasks to be automated by short-lifetime AIs, rather than long, individual lifetime AIs, and I’m not nearly as confident as you or @1a3orn that long-term memory isn’t necessary and not having it pays a crippling capability tax.