...our support base is (understandably) anxious to see a shift from movement-building work to FAI research, a shift I have been fighting for since I was made Executive Director.
This raises several questions for me:
How do you divide movement building/FAI research? What is the Singularity Summit? What is “writing papers like the two for the Singularity Hypothesis volume”? I worry about these questions because it seems to me that so far, the Singularity Institute’s big successes are in things that could be considered “movement building,” and I worry moving away from those could actually make SIAI less effective.
What the expectations of the donor base? In particular, do a substantial number of them feel that the main reason to donate to SIAI is in hopes that SIAI will develop FAI relatively soon? I worry about these questions because I worry about SIAI being in the position of having to appease donors with unrealistic expectations.
Speaking for myself, as a donor, I expect SI to “do the right thing, whatever that may be, to achieve a positive result from greater than human intelligence”. I don’t intend to dictate their specific actions; they’re the experts.
I like to see some object level FAI research to ground meta level strategies like movement building. But movement building is very important right now, as SI needs to recruit and fund additional FAI researchers. Publishing a sequence on open problems in FAI seems like a very well grounded form of movement building.
My expectation on timeframes is a wide distribution. I am uncertain about how much work is needed to solve the problem, and how fast SI will be able to recruit researchers.
An explicit list of open problems seems closer to the metal of object level progress than, say, a better website. (I’m not implying that the less object level things aren’t very worth doing—they make the object level work possible).
This raises several questions for me:
How do you divide movement building/FAI research? What is the Singularity Summit? What is “writing papers like the two for the Singularity Hypothesis volume”? I worry about these questions because it seems to me that so far, the Singularity Institute’s big successes are in things that could be considered “movement building,” and I worry moving away from those could actually make SIAI less effective.
What the expectations of the donor base? In particular, do a substantial number of them feel that the main reason to donate to SIAI is in hopes that SIAI will develop FAI relatively soon? I worry about these questions because I worry about SIAI being in the position of having to appease donors with unrealistic expectations.
Speaking for myself, as a donor, I expect SI to “do the right thing, whatever that may be, to achieve a positive result from greater than human intelligence”. I don’t intend to dictate their specific actions; they’re the experts.
I like to see some object level FAI research to ground meta level strategies like movement building. But movement building is very important right now, as SI needs to recruit and fund additional FAI researchers. Publishing a sequence on open problems in FAI seems like a very well grounded form of movement building.
My expectation on timeframes is a wide distribution. I am uncertain about how much work is needed to solve the problem, and how fast SI will be able to recruit researchers.
An explicit list of open problems seems closer to the metal of object level progress than, say, a better website. (I’m not implying that the less object level things aren’t very worth doing—they make the object level work possible).