From working at Google from 2013 to 2017… I feel this was structurally very predictable. Google, as I experienced it, was a BIG moral maze. I left them to be the ethicist of a blockchain project, and it started out well, but by the time I resigned from that I felt that it had become… a small moral maze.
However in the first case I lacked power, and in the second case I lacked the wisdom to treat this as a first class problem (and, indeed, when the blockchain started getting maze-ish, it was before LW even had the sequence on the subject, and before I’d read the Moral Maze book). Now I’m looking at founding for myself, so that I will have power and wisdom both, but I’m interested in cultural clarity here, because I don’t think any single person can make such a result happen. (For example, look at OpenAI, and the board’s attempt to remove Sam Altman via normal procedures, which was irregularly reversed.)
I respect that you attempted to do what you did, but I’m very curious if you learned larger lessons that could be applied to the same issue in a way that actually causes the structural failure to not happen in general, due EITHER to trustless institutional forms, OR to simply picking leaders who aren’t ethical trashhats… OR something else?
Anything that actually works seems like it would be good, but I don’t have a clean plan to that I know will reliably work, I just have goals and hunches right now...
If you made a list of “five lessons for how to cause the technology one produces to not be used by stupid people for evil ends”, what would be on it?
From working at Google from 2013 to 2017… I feel this was structurally very predictable. Google, as I experienced it, was a BIG moral maze. I left them to be the ethicist of a blockchain project, and it started out well, but by the time I resigned from that I felt that it had become… a small moral maze.
However in the first case I lacked power, and in the second case I lacked the wisdom to treat this as a first class problem (and, indeed, when the blockchain started getting maze-ish, it was before LW even had the sequence on the subject, and before I’d read the Moral Maze book). Now I’m looking at founding for myself, so that I will have power and wisdom both, but I’m interested in cultural clarity here, because I don’t think any single person can make such a result happen. (For example, look at OpenAI, and the board’s attempt to remove Sam Altman via normal procedures, which was irregularly reversed.)
I respect that you attempted to do what you did, but I’m very curious if you learned larger lessons that could be applied to the same issue in a way that actually causes the structural failure to not happen in general, due EITHER to trustless institutional forms, OR to simply picking leaders who aren’t ethical trashhats… OR something else?
Anything that actually works seems like it would be good, but I don’t have a clean plan to that I know will reliably work, I just have goals and hunches right now...
If you made a list of “five lessons for how to cause the technology one produces to not be used by stupid people for evil ends”, what would be on it?