The ELK paper is long but I’ve found it worthwhile, and after spending a bit of time noodling on it — one of my takeaways is I think this is essentially a failure mode for the approaches to factored cognition I’ve been interested in. (Maybe it’s a failure mode in factored cognition generally.
I expect that I’ll want to spend more time thinking about ELK-like problems before spending a bunch more time thinking about factored cognition.
In particular it’s now probably a good time to start separating a bunch of things I had jumbled together, namely:
Developing AI technology that helps us do alignment research
Developing aligned AI
Previously I had hoped that the two would be near each other in ways that permit progress on both at the same time.
Now I think without solving ELK I would want to be more careful and intentional about how/when to develop AI tech to help with alignment.
The ELK paper is long but I’ve found it worthwhile, and after spending a bit of time noodling on it — one of my takeaways is I think this is essentially a failure mode for the approaches to factored cognition I’ve been interested in. (Maybe it’s a failure mode in factored cognition generally.
I expect that I’ll want to spend more time thinking about ELK-like problems before spending a bunch more time thinking about factored cognition.
In particular it’s now probably a good time to start separating a bunch of things I had jumbled together, namely:
Developing AI technology that helps us do alignment research
Developing aligned AI
Previously I had hoped that the two would be near each other in ways that permit progress on both at the same time.
Now I think without solving ELK I would want to be more careful and intentional about how/when to develop AI tech to help with alignment.