Like, members of those groups do not regularly sit down and make extensive plans about how to optimize other people’s beliefs in the same way as seems routine around here.
I’ve been around the community for 10 years. I don’t think I’ve ever seen this?[1]
Am I just blind to this? Am I seeing it all the time, except I have lower standards what should “count”? Am I just selected out of such conversations somehow?
I currently work for an org that is explicitly focused on communicating the AI situation to the world, and to policymakers in particular. We are definitely attempting to be strategic about that, and we put a hell of a lot of effort into doing it well (eg running many many test sessions, where we try to explain what’s up to volunteers, see what’s confusing, and adjust what we’re saying).
(Is this the kind of thing you mean?)
But, importantly, we’re clear about trying to frankly communicate our actual beliefs, including our uncertainties, and are strict about adhering to standards of local validity and precise honesty: I’m happy to talk with you about the confusing experimental results that weaken our high level claims (though admittedly, under normal time constraints, I’m not going to lead with that).
Pretty much every day, I check “If someone had made this argument against [social media], would that have made me think that it was imperative to shut it down?”, about proffered anti AI arguments.
I’ve been around the community for 10 years. I don’t think I’ve ever seen this?
Also, come on, this seems false. I am pretty sure you’ve seen Leverage employees do this, and my guess is you’ve seen transcripts of chats of this happening with quite a lot of agency at FTX with regards to various auditors and creditors.
I currently work for an org that is explicitly focused on communicating the AI situation to the world, and to policymakers in particular. We are definitely attempting to be strategic about that, and we put a hell of a lot of effort into doing it well (eg running many many test sessions, where we try to explain what’s up to volunteers, see what’s confusing, and adjust what we’re saying).
Yeah, that does sound roughly like what I mean, and then I think most people just drop the second part:
But, importantly, we’re clear about trying to frankly communicate our actual beliefs, including our uncertainties, and are strict about adhering to standards of local validity and precise honesty: I’m happy to talk with you about the confusing experimental results that weaken our high level claims (though admittedly, under normal time constraints, I’m not going to lead with that).
I do not think that SBF was doing this part. He was doing the former though!
Am I just blind to this? Am I seeing it all the time, except I have lower standards what should “count”? Am I just selected out of such conversations somehow?
My best guess you are doing a mixture of:
Indeed self-selecting yourself out of these environments
Having a too-narrow conception of the “AI Safety community” that forms a Motte where you conceptually exclude people who do this a lot (e.g. the labs themselves), but in a way that then makes posts like the OP we are commenting on misleading
Probably have somewhat different standards for this (indeed, a thing I’ve updated on over the years is that a lot of powerful optimization can happen here between people, where e.g. one party sets up a standard in good-faith, and then another party starts goodharting on that standard in largely good-faith, and the end-result is a lot of deception).
indeed, a thing I’ve updated on over the years is that a lot of powerful optimization can happen here between people, where e.g. one party sets up a standard in good-faith, and then another party starts goodharting on that standard in largely good-faith, and the end-result is a lot of deception
Do you have an example of this? (It sounds like you think that I might be participating in this dynamic on one side or the other.)
I think this is roughly what happened when FTX was spending a huge amount of money before it all collapsed and a lot of people started new projects under pretty dubious premises to look appealing to them. I also think this is still happening quite a lot around OpenPhil, with a lot of quite bad research being produced, and a lot of people digging themselves into holes (and also trying to enforce various norms that don’t really make sense, but where they think if they enforce it, they are more likely to get money, which does unfortunately work).
I’ve been around the community for 10 years. I don’t think I’ve ever seen this?[1]
Am I just blind to this? Am I seeing it all the time, except I have lower standards what should “count”? Am I just selected out of such conversations somehow?
I currently work for an org that is explicitly focused on communicating the AI situation to the world, and to policymakers in particular. We are definitely attempting to be strategic about that, and we put a hell of a lot of effort into doing it well (eg running many many test sessions, where we try to explain what’s up to volunteers, see what’s confusing, and adjust what we’re saying).
(Is this the kind of thing you mean?)
But, importantly, we’re clear about trying to frankly communicate our actual beliefs, including our uncertainties, and are strict about adhering to standards of local validity and precise honesty: I’m happy to talk with you about the confusing experimental results that weaken our high level claims (though admittedly, under normal time constraints, I’m not going to lead with that).
Pretty much every day, I check “If someone had made this argument against [social media], would that have made me think that it was imperative to shut it down?”, about proffered anti AI arguments.
Also, come on, this seems false. I am pretty sure you’ve seen Leverage employees do this, and my guess is you’ve seen transcripts of chats of this happening with quite a lot of agency at FTX with regards to various auditors and creditors.
Yeah, that does sound roughly like what I mean, and then I think most people just drop the second part:
I do not think that SBF was doing this part. He was doing the former though!
My best guess you are doing a mixture of:
Indeed self-selecting yourself out of these environments
Having a too-narrow conception of the “AI Safety community” that forms a Motte where you conceptually exclude people who do this a lot (e.g. the labs themselves), but in a way that then makes posts like the OP we are commenting on misleading
Probably have somewhat different standards for this (indeed, a thing I’ve updated on over the years is that a lot of powerful optimization can happen here between people, where e.g. one party sets up a standard in good-faith, and then another party starts goodharting on that standard in largely good-faith, and the end-result is a lot of deception).
Do you have an example of this? (It sounds like you think that I might be participating in this dynamic on one side or the other.)
I think this is roughly what happened when FTX was spending a huge amount of money before it all collapsed and a lot of people started new projects under pretty dubious premises to look appealing to them. I also think this is still happening quite a lot around OpenPhil, with a lot of quite bad research being produced, and a lot of people digging themselves into holes (and also trying to enforce various norms that don’t really make sense, but where they think if they enforce it, they are more likely to get money, which does unfortunately work).