Philosophy PhD student, worked at AI Impacts, then Center on Long-Term Risk, then OpenAI. Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI. Not sure what I’ll do next yet. Views are my own & do not represent those of my current or former employer(s). I subscribe to Crocker’s Rules and am especially interested to hear unsolicited constructive criticism. http://sl4.org/crocker.html
Some of my favorite memes:
(by Rob Wiblin)
My EA Journey, depicted on the whiteboard at CLR:
(h/t Scott Alexander)
Can you please link me to more on this? I was under the impression that pascal’s mugging happens for any utility function that grows at least as fast as the probabilities shrink, and the probabilities shrink exponentially for normal probability functions. (For example: In the toy model of the St. Petersburg problem, the utility function grows exactly as fast as the probability function shrinks, resulting in infinite expected utility for playing the game.)
Also: As I understand them, utility functions aren’t of the form “I want to see X P often and Y 1-P often.” They are more like “X has utility 200, Y has utility 150, Z has utility 24...” Maybe the form you are talking about is a special case of the form I am talking about, but I don’t yet see how it could be the other way around. As I’m thinking of them, utility functions aren’t about what you see at all. They are just about the world. The point is, I’m confused by your explanation & would love to read more about this.