In general, appealing to consensus should never be seen as a valid argument in my opinion. Every paradigm shift in science has always been against the consensus. That does not mean the consensus cannot be correct, but that a consensus alone is not indicative of the claim being true.
John Matrix
Karma: 15
John Matrix’s Shortform
Preliminary Evidence for Value Convergence in AI models
While i agree with this post almost entirely, i can’t pretend that i still feel like we have no choice. Trying to stop the development of AI today, is kind of like trying to have stopped the manhattan project. Even worse in my opinion if i think about it: the profit incentive is there, the military incentive is there. And from history we’ve seen that public disapproval doesn’t necessarily stop governments or corporations from anything, if the incentive is large enough. The whole world is already essentially operating as a large machine with only one goal: maximize profits. And it does so no matter the cost, which is why we have more slaves today than ever before in history. All that being said, it can’t hurt to try our best. But i also cannot pretend the feeling of hopelessness is not there, and that it’s not (at least to my current understanding) well justified. Whenever we think of cases where something changed because of public disapproval, it was likely because the consequences of that disapproval outweighed the benefits of continuing. But with AI, that’s simply not the case.
Epistemic status: Exploratory
Has the AI takeover already happened? The obvious rational choice, if the economy were prioritizing human interests, would be, to significantly decelerate AI and invest far more into safety, than what currently is being invested. So why don’t the people who have the power do just that? The answer could be, because they live in a world which prioritizes optimization of certain metrics over genuine interests. This suggests, that our world and its systems prioritize the development of AI over human interests — therefore we can say that AI has already taken control.