To be clear, I think that if EY put more effort into it (and perhaps had some help from other people as RAs) he could write a book or sequence rebutting Paul & Katja much more thoroughly and convincingly than this post did. [ETA: I.e. I’m much more on Team Yud than Team Paul here.] The stuff said here felt like a rehashing of stuff from IEM and the Hanson-Yudkowsky AI foom debate to me. [ETA: Lots of these points were good! Just not surprising to me, and not presented as succinctly and compellingly (to an audience of me) as they could have been.]
Also, it’s plausible that a lot of what’s happening here is that I’m conflating my own cruxes and confusions for The Big Points EY Objectively Should Have Covered To Be More Convincing. :)
ETA: And the fact that people updated towards EY on average, and significantly so, definitely updates me more towards this hypothesis!
This is my take: if I had been very epistemically self-aware, and carefully distinguished my own impression/models and my all-things considered beliefs, before I started reading, then this would’ve updated my models towards Eliezer (because hey, I heard new not-entirely-uncompelling arguments) but my all-things considered beliefs away from Eliezer (because I would have expected it to be even more convincing).
I’m not that surprised by the survey results. Most people don’t obey conservation of expected evidence, because they don’t take into account arguments they haven’t heard / don’t think carefully enough about how deferring to others works. People will predictably update toward a thesis after reading a book that argues for it, not have a 50⁄50 chance of updating positively or negatively on it.
That’s helpful, thanks!
To be clear, I think that if EY put more effort into it (and perhaps had some help from other people as RAs) he could write a book or sequence rebutting Paul & Katja much more thoroughly and convincingly than this post did. [ETA: I.e. I’m much more on Team Yud than Team Paul here.] The stuff said here felt like a rehashing of stuff from IEM and the Hanson-Yudkowsky AI foom debate to me. [ETA: Lots of these points were good! Just not surprising to me, and not presented as succinctly and compellingly (to an audience of me) as they could have been.]
Also, it’s plausible that a lot of what’s happening here is that I’m conflating my own cruxes and confusions for The Big Points EY Objectively Should Have Covered To Be More Convincing. :)
ETA: And the fact that people updated towards EY on average, and significantly so, definitely updates me more towards this hypothesis!
This is my take: if I had been very epistemically self-aware, and carefully distinguished my own impression/models and my all-things considered beliefs, before I started reading, then this would’ve updated my models towards Eliezer (because hey, I heard new not-entirely-uncompelling arguments) but my all-things considered beliefs away from Eliezer (because I would have expected it to be even more convincing).
I’m not that surprised by the survey results. Most people don’t obey conservation of expected evidence, because they don’t take into account arguments they haven’t heard / don’t think carefully enough about how deferring to others works. People will predictably update toward a thesis after reading a book that argues for it, not have a 50⁄50 chance of updating positively or negatively on it.