I’m currently going through the books Modal Logic by Blackburn et al. and Dynamic Epistemic Logic by Ditmarsch et al. Both of these books seem to me potentially useful for research on AI Alignment, but I’m struggling to find any discourse on LW about it. If I’m simply missing it, could someone point me to it? Otherwise, does anyone have an idea as to why this kind of research is not done? (Besides the “there are too few people working on AI alignment in general” answer).
We had a bit more usage of the formalism of those theories in the 2010s, like using modal logics to investigate co-operation/defection in logical decision theories. As for Dynamic Epistemic logic, well, the blurb does make it look sort of relevant.
Perhaps it might have something interesting to say on the tiling agents problem, or on decision theory, or so on. But other things have looked superficially relevant in the past, too. E.g. fuzzy logics, category theory, homotopy type theory etc. And AFAICT, no one has really done anything that really used the practical tools of these theories to make any legible advances. And of what was legibly impressive, it didn’t seem to be due to the machinery of those theories, but rather the cleverness of the people using them. Likewise for the past work in alignment using modal logics.
So I’m not sure what advantage you’re seeing here, because I haven’t read the books and don’t have the evidence you do. But my priors are that if you have any good ideas about how to make progress in alignment, it’s not going to be downstream of using the formalism in the books you mentioned.
Thanks for the information, I’ll look into this some more based on what you mentioned.
So I’m not sure what advantage you’re seeing here, because I haven’t read the books and don’t have the evidence you do. But my priors are that if you have any good ideas about how to make progress in alignment, it’s not going to be downstream of using the formalism in the books you mentioned.
I didn’t have any particular new ideas about how to make progress in alignment, but rather felt as though the framework of these books provide an interesting lens to model systems and agents that could be of interest, and subsequently prove various properties that are necessary/faborable. It’s helpful that your priors say these won’t be downstream of using the formalisms in the mentioned books; it may rather be a phenomenon of me not being adequately familiar with formal frameworks.
felt as though the framework of these books provide an interesting lens to model systems and agents that could be of interest, and subsequently prove various properties that are necessary/faborable
Your feelings might be right! I don’t have a not a strong prior, and in general I’d say that people should follow their inner compass and work on what they’re excited about. It’s very hard to convey your illegible intuitions to others, and all too easy for social pressure to squash them. Not sure what someone should really do in this situation, beyond keeping your eyes on the hard problems of alignment and finding ways to get feedback from reality on your ideas as fast as possible.
I’m currently going through the books Modal Logic by Blackburn et al. and Dynamic Epistemic Logic by Ditmarsch et al. Both of these books seem to me potentially useful for research on AI Alignment, but I’m struggling to find any discourse on LW about it. If I’m simply missing it, could someone point me to it? Otherwise, does anyone have an idea as to why this kind of research is not done? (Besides the “there are too few people working on AI alignment in general” answer).
We had a bit more usage of the formalism of those theories in the 2010s, like using modal logics to investigate co-operation/defection in logical decision theories. As for Dynamic Epistemic logic, well, the blurb does make it look sort of relevant.
Perhaps it might have something interesting to say on the tiling agents problem, or on decision theory, or so on. But other things have looked superficially relevant in the past, too. E.g. fuzzy logics, category theory, homotopy type theory etc. And AFAICT, no one has really done anything that really used the practical tools of these theories to make any legible advances. And of what was legibly impressive, it didn’t seem to be due to the machinery of those theories, but rather the cleverness of the people using them. Likewise for the past work in alignment using modal logics.
So I’m not sure what advantage you’re seeing here, because I haven’t read the books and don’t have the evidence you do. But my priors are that if you have any good ideas about how to make progress in alignment, it’s not going to be downstream of using the formalism in the books you mentioned.
Thanks for the information, I’ll look into this some more based on what you mentioned.
I didn’t have any particular new ideas about how to make progress in alignment, but rather felt as though the framework of these books provide an interesting lens to model systems and agents that could be of interest, and subsequently prove various properties that are necessary/faborable. It’s helpful that your priors say these won’t be downstream of using the formalisms in the mentioned books; it may rather be a phenomenon of me not being adequately familiar with formal frameworks.
Your feelings might be right! I don’t have a not a strong prior, and in general I’d say that people should follow their inner compass and work on what they’re excited about. It’s very hard to convey your illegible intuitions to others, and all too easy for social pressure to squash them. Not sure what someone should really do in this situation, beyond keeping your eyes on the hard problems of alignment and finding ways to get feedback from reality on your ideas as fast as possible.
Good that you mention this, will keep that mind!
Some links on modal logic for FDT-style decision theory and coordination:
Robust Cooperation in the Prisoner’s Dilemma
Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents
Cooperative and uncooperative institution designs: Surprises and problems in open-source game theory
Modal Fixpoint Cooperation without Löb’s Theorem
Comparing Payor & Löb
Thank you!