Surprise at the quantity of work that had gone into it.
Alas, I totally failed to see the claimed “strange singularity at the heart of decision theory”.
My favourite bit was probably the speculations about agent boundaries—starting on p.108. Alas, from my POV, no mention of the wirehead problem.
Update 2011-06-26 regarding the new version. The bit that reads:
This manuscript was cut off here, but interested readers are suggested to look at these sources for more discussion:
...seems to have been deleted, and 3 pages worth of references have been added. The document seems to have had negligible additions, though—the bit on p.108 has moved back onto page 107. There seem to be a few more extra lines at the end about how “change” is a harmful concept in decision theory.
I scanned it. My initial reactions:
Surprise that the document existed;
TL;DR;
Surprise at the quantity of work that had gone into it.
Alas, I totally failed to see the claimed “strange singularity at the heart of decision theory”.
My favourite bit was probably the speculations about agent boundaries—starting on p.108. Alas, from my POV, no mention of the wirehead problem.
Update 2011-06-26 regarding the new version. The bit that reads:
...seems to have been deleted, and 3 pages worth of references have been added. The document seems to have had negligible additions, though—the bit on p.108 has moved back onto page 107. There seem to be a few more extra lines at the end about how “change” is a harmful concept in decision theory.