It took a lot of effort to write, something like 3 days of my time. Distillation is hard.
Most of this effort was not in understanding the original post (took me 2-3 hours to understand the math)
I sent drafts to johnwentworth several times and had several conversations with him to refine this piece. This probably spent ~2 hours of his time.
I’m not satisfied with the final result. It seems like the point the original post made was fairly obvious and I used way too many words to explain it properly. Maybe John thought the interpretation of the math was fairly deep and I thought it wasn’t very deep?
I think that since John is a good and prolific writer already compared to most alignment researchers, there is higher value in distilling ideas of other researchers. It’s hard to produce a lot of value from content already on LW.
Paul Christiano blogposts are somewhat famously opaque; distillations of these have worked in the past and still seem pretty valuable. The highest-relevance academic papers might be better. But many of the highest-value distillations probably involve talking to researchers to get things they’re too busy to write down at all.
Thoughts on the process of writing this post:
It took a lot of effort to write, something like 3 days of my time. Distillation is hard.
Most of this effort was not in understanding the original post (took me 2-3 hours to understand the math)
I sent drafts to johnwentworth several times and had several conversations with him to refine this piece. This probably spent ~2 hours of his time.
I’m not satisfied with the final result. It seems like the point the original post made was fairly obvious and I used way too many words to explain it properly. Maybe John thought the interpretation of the math was fairly deep and I thought it wasn’t very deep?
I think that since John is a good and prolific writer already compared to most alignment researchers, there is higher value in distilling ideas of other researchers. It’s hard to produce a lot of value from content already on LW.
Paul Christiano blogposts are somewhat famously opaque; distillations of these have worked in the past and still seem pretty valuable. The highest-relevance academic papers might be better. But many of the highest-value distillations probably involve talking to researchers to get things they’re too busy to write down at all.