A general problem in this area is that current AI training have way worse data efficiency than humans, but “make up for it” by being able to train on a lot more data very quickly. As a result, AIs can become superhuman in areas where we can give them lots of data or feedback, like games, math, coding, but lag behind in other areas like philosophy and long-horizon real-world strategy. Does “good epistemics” depend on changing this dynamic, such that AIs become at least as data efficient as humans (which seems scary from a capabilities/timelines perspective), or do you see it as a potentially independent project or approach?
To put it another way, can we achieve “good epistemics” in data-scarce fields, without greatly increasing AI capabilities in general? If not, how do we ensure that people working on “good epistemics” don’t succeed before we’re ready for it, in other areas like AI alignment?
Hi Wei, my apologies for my late reply, I only saw your comment after returning to this post when I saw that Anthropic mentioned in their recent blog post that perhaps models will be sufficiently wise to not attempt RSI.
I agree that working on good epistemics, to outpace capability improvements in general, would require 1.) changing the dynamic, 2.) significant investment in projects to curate data / feedback on good fuzzy reasoning, or 3.) using extensive scaffolding (aka ‘meta systems’) to elicit better epistemics from models with jagged performance in these domains. As an example I think the competition that FLF is running here might lend itself to prototypes of better meta-systems, which could ladder up to better epistemics. (I’m also excited about 2 though I don’t have ready examples of what that might look like)
A general problem in this area is that current AI training have way worse data efficiency than humans, but “make up for it” by being able to train on a lot more data very quickly. As a result, AIs can become superhuman in areas where we can give them lots of data or feedback, like games, math, coding, but lag behind in other areas like philosophy and long-horizon real-world strategy. Does “good epistemics” depend on changing this dynamic, such that AIs become at least as data efficient as humans (which seems scary from a capabilities/timelines perspective), or do you see it as a potentially independent project or approach?
To put it another way, can we achieve “good epistemics” in data-scarce fields, without greatly increasing AI capabilities in general? If not, how do we ensure that people working on “good epistemics” don’t succeed before we’re ready for it, in other areas like AI alignment?
Hi Wei, my apologies for my late reply, I only saw your comment after returning to this post when I saw that Anthropic mentioned in their recent blog post that perhaps models will be sufficiently wise to not attempt RSI.
I agree that working on good epistemics, to outpace capability improvements in general, would require 1.) changing the dynamic, 2.) significant investment in projects to curate data / feedback on good fuzzy reasoning, or 3.) using extensive scaffolding (aka ‘meta systems’) to elicit better epistemics from models with jagged performance in these domains. As an example I think the competition that FLF is running here might lend itself to prototypes of better meta-systems, which could ladder up to better epistemics. (I’m also excited about 2 though I don’t have ready examples of what that might look like)