Sure, but why do you think that means they had a positive impact? Even if alignment turns out to be easy instead of hard, that doesn’t seem like it’s evidence that Lightcone had a positive impact.
[I agree a simple “alignment hard → Lightcone bad” model gets contradicted by it, but that’s not how I read their model.]
My reading is that Noosphere89 thinks that Lightcone has helped in bringing in/upskiling a number of empirical/prosaic alignment researchers. In worlds where alignment is relatively easy, this is net positive as the alignment benefits are higher than the capabilities costs, while in worlds where alignment is very hard, we might expect the alignment benefits to be marginal while the capabilities costs continue to be very real.
I did argue that closing the Lightcone offices was the right thing, but my point is that part of the reasoning relies on a core assumption that AI Alignment isn’t very iterable and will generally cost capabilities that I find probably false.
I am open to changing my mind, but I see a lot of reasoning on AI Alignment that is kinda weird to me by Habryka and Ben Pace.
Sure, but why do you think that means they had a positive impact? Even if alignment turns out to be easy instead of hard, that doesn’t seem like it’s evidence that Lightcone had a positive impact.
[I agree a simple “alignment hard → Lightcone bad” model gets contradicted by it, but that’s not how I read their model.]
My reading is that Noosphere89 thinks that Lightcone has helped in bringing in/upskiling a number of empirical/prosaic alignment researchers. In worlds where alignment is relatively easy, this is net positive as the alignment benefits are higher than the capabilities costs, while in worlds where alignment is very hard, we might expect the alignment benefits to be marginal while the capabilities costs continue to be very real.
I did argue that closing the Lightcone offices was the right thing, but my point is that part of the reasoning relies on a core assumption that AI Alignment isn’t very iterable and will generally cost capabilities that I find probably false.
I am open to changing my mind, but I see a lot of reasoning on AI Alignment that is kinda weird to me by Habryka and Ben Pace.