That said, It also appears to me that Eliezer is probably not the most careful reasoner, and appears indeed often (perhaps egregiously) overconfident. That doesn’t mean one should begrudge people finding value in the sequences although it is certainly not ideal if people take them as mantras rather than useful pointers and explainers for basic things (I didn’t read them, so might have an incorrect view here). There does appear to be some tendency to just link to some point made in the sequences as some airtight thing, although I haven’t found it too pervasive recently.
Amalthea
You’re describing a situational character flaw which doesn’t really have any bearing on being able to reason carefully overall.
I’m echoing other commenters somewhat, but—personally—I do not see people being down-voted simply for having different viewpoints. I’m very sympathetic to people trying to genuinely argue against “prevailing” attitudes or simply trying to foster a better general understanding. (E.g. I appreciate Matthew Barnett’s presence, even though I very much disagree with his conclusions and find him overconfident). Now, of course, the fact that I don’t notice the kind of posts you say are being down-voted may be because they are sufficiently filtered out, which indeed would be undesirable from my perspective and good to know.
When you have a role in policy or safety, it may usually be a good idea not to voice strong opinions on any given company. If you nevertheless feel compelled to do so by circumstances, it’s a big deal if you have personal incentives against that—especially if they’re not disclosed.
Might be good to estimate the date of the recommendation—as the interview where Carmack mentioned this was in 2023, a rough guess might be 2021/22?
It might not be legal reasons specifically, but some hard-to-specify mix of legal reasons/intimidation/bullying. While it’s useful to discuss specific ideas, it should be kept in mind that Altman doesn’t need to restrict his actions to any specific avenue that could be neatly classified.
I’d like to listen to something like this in principle, but it has really unfortunate timing with the further information that’s been revealed, making it somewhat less exciting. It would be interesting to hear how/whether the participants believes change.
Have you ever written anything about why you hate the AI safety movement? I’d be quite curious to hear your perspective.
I think the best bet is to vote for a generally reasonable party. Despite their many flaws, it seems like Green Party or SPD are the best choices right now. (CDU seems to be too influenced in business interests, the current FDP is even worse)
The alternative would be to vote for a small party with a good agenda to help signal-boost them, but I don’t know who’s around these days.
It’s not an entirely unfair characterization.
Half a year ago, I’d have guessed that OpenAI leadership, while likely misguided, was essentially well-meaning and driven by a genuine desire to confront a difficult situation. The recent series of events has made me update significantly against the general trustworthiness and general epistemic reliability of Altman and his circle. While my overall view of OpenAI’s strategy hasn’t really changed, my likelihood of them possibly “knowing better” has dramatically gone down now.
Fundamentally, OP is making the case that Biorisk is an extremely helpful (but not exact) analogy for AI risk, in the sense that we can gain understanding by looking the ways it’s analogous, and then get an even more nuanced understanding by analyzing the differences.
The point made seems to be more about it’s place in the discourse than about the value of the analogy itself? (E.g. “The Biorisk analogy is over-used” would be less misleading then)
What do you think about building legal/technological infrastructure to enable a prompt pause, should it seem necessary?
I agree that potentially the benefits can go to everyone. The point is that as the person pursuing AGI you are making the choice for everyone else.
The asymmetry is that if you do something that creates risk for everyone else, I believe that does single you out as an aggressor? While conversely, enforcing norms that prevent such risky behavior seems justified. The fact that by default people are mortal is tragic, but doesn’t have much bearing here. (You’d still be free to pursue life-extension technology in other ways, perhaps including limited AI tools).
Ideally, of course, there’d be some sort of democratic process here that let’s people in aggregate make informed (!) choices. In the real world, it’s unclear what a good solution here would be. What we have right now is the big labs creating facts that society has trouble catching up with, which I think many people are reasonably uncomfortable with.
I think the perspective that you’re missing regarding 2. is that by building AGI one is taking the chance of non-consensually killing vast amounts of people and their children for some chance of improving one’s own longevity.
Even if one thinks it’s a better deal for them, a key point is that you are making the decision for them by unilaterally building AGI. So in that sense it is quite reasonable to see it as an “evil” action to work towards that outcome.
Somewhat of a nitpick, but the relevant number would be p(doom | strong AGI being built) (maybe contrasted with p(utopia | strong AGI)) , not overall p(doom).
I was down voting this particular post because I perceived it as mostly ideological and making few arguments, only stating strongly that government action will be bad. I found the author’s replies in the comments much more nuanced and would not have down-voted if I’d perceived the original post to be of the same quality.
Basically, I think whether or not one thinks whether alignment is hard or not is much more of the crux than whether or not they’re utilitarian.
Pesonally, I don’t find Pope & Belrose very convincing, although I do commend them for the reasonable effort—but if I did believe that AI is likely to go well, I’d probably also be all for it. I just don’t see how this is related to utilitarianism (maybe for all but a very small subset of people in EA).
“One reason that goes overlooked is that most human beings are not utilitarians” I think this point is just straightforwardly wrong. Even from a purely selfish perspective, it’s reasonable to want to stop AI.
The main reason humanity is not going to stop seems mainly like coordination problems, or something close to learned helplessness in these kind of competitive dynamics.
I agree with the first sentence. I agree with the second sentence with the caveat that it’s not strong absolute evidence, but mostly applies to the given setting (which is exactly what I’m saying).
People aren’t fixed entities and the quality of their contributions can vary over time and depend on context.