Elad Hazan
Karma: 22
Alignment as Equilibrium Design
A course I tought on AI safety and alignment at Princeton from 2 years ago:
https://sites.google.com/view/cos598aisafety/list of materials is on the webpage, happy to share lecture slides as well.
I believe Peter Henderson has a newer version. Roger Grosse taught a similar course at U Toronto.
Nice post! Another place where nonlinear utilities have been considered is in technical game theory / regret minimization literature. A consideration for introducing them is computational complexity—while VNM utility applies, it may be inefficient to linearize the utilities in terms of representation and downstream algorithms. An example is here, where a log-utility game is considered due to complexity/efficiency considerations.