An anonymous user named Omega posted a critique of Redwood on the EA Forum. The post highlights four main areas: (1) Lack of senior ML staff, (2) Lack of communication & engagement with the ML community, (3) Underwhelming research output, and (4) Work culture issues.
I’m linkposting it here, since I imagine some LW users will have thoughts/comments. See also this comment from Nate Thomas, and note that Redwood has an anonymous feedback form.
We believe that Redwood has some serious flaws as an org, yet has received a significant amount of funding from a central EA grantmaker (Open Philanthropy). Inadequately kept in check conflicts of interest (COIs) might be partly responsible for funders giving a relatively immature org lots of money and causing some negative effects on the field and EA community. We will share our critiques of Constellation (and Open Philanthropy) in a follow-up post. We also have some suggestions for Redwood that we believe might help them achieve their goals.
Redwood is a young organization that has room to improve. While there may be flaws in their current approach, it is possible for them to learn and adapt in order to produce more accurate and reliable results in the future. Many successful organizations made significant pivots while at a similar scale to Redwood, and we remain cautiously optimistic about Redwood’s future potential.
Standard caveat that I don’t agree with everything in the post or even endorse its main conclusions; also see my comment.
Copying over my comment from the EA Forum version.
I think it’s great that you’re releasing some posts that criticize/red-team some major AIS orgs. It’s sad (though understandable) that you felt like you had to do this anonymously.
I’m going to comment a bit on the Work Culture Issues section. I’ve spoken to some people who work at Redwood, have worked at Redwood, or considered working at Redwood.
I think my main comment is something like you’ve done a good job pointing at some problems, but I think it’s pretty hard to figure out what should be done about these problems. To be clear, I think the post may be useful to Redwood (or the broader community) even if you only “point at problems”, and I don’t think people should withhold these write-ups unless they’ve solved all the problems.
But in an effort to figure out how to make these critiques more valuable moving forward, here are some thoughts:
If I were at Redwood, I would probably have a reaction along the lines of “OK, you pointed out a list of problems. Great. We already knew about most of these. What you’re not seeing is that there are also 100 other problems that we are dealing with: lack of management experience, unclear models of what research we want to do, an ever-evolving AI progress landscape, complicated relationships we need to maintain, interpersonal problems, a bunch of random ops things, etc. This presents a tough bind: on one hand, we see some problems, and we want to fix them. On the other hand, we don’t know any easy ways to fix them that don’t trade-off against other extremely important priorities.”
As an example, take the “intense work culture” point. The most intuitive reaction is “make the work culture less intense—have people work fewer hours.” But this plausibly has trade-offs with things like research output. You could make the claim that “on the margin, if Redwood employees worked 10 fewer hours per week, we expect Redwood would be more productive in the long-run because of reduced burnout and a better culture”, but this is a substantially different (and more complicated) claim to make. And it’s not obviously-true.
As another example, take the “people feel pressure to defer” point. I personally agree that this is a big problem for Redwood/Constellation/the Bay Area scene. My guess is Buck/Nate/Bill agree. It’s possible that they don’t think it’s a huge deal relative to the other 100 things on their plate. And maybe they’re wrong about that, but I think that needs to be argued for if you want them to prioritize it. Alternatively, the problem might be that they simply don’t know what to do. Like, maybe they could put up a sign that says “please don’t defer—speak your mind!” Or maybe they could say “thank you” more when people disagree, or something. But I think often the problem is that people don’t know what interventions would be able to fix well-known problems (again, without trading off against something else that is valuable).
I’m also guessing that there are some low-hanging fruit interventions that external red-teamers could identify. For example, here are three things that I think Redwood should do:
Hire a full-time productivity coach/therapist for the Constellation offices. (I recommended this to Nate many months ago. He seemed to (correctly, imo) predict that burnout would be a big problem for Redwood employees, and he said he’d think about the therapist/coach suggestion. I believe they haven’t hired one.)
Hire an external red-teamer to interview current and former employees, identify work culture issues, and identify interventions to improve things. Conditional on this person/team identifying useful (and feasible) interventions, work with leadership to actually get them implemented. (I’m not sure if they’re doing this, and also maybe your group is already doing this, but the post focused on problems rather than interventions?)
Have someone red-team communications around employee expectations, work-trial expectations, and expectation-setting during the onboarding process. I think I’m fine with some people opting-in to a culture that expects them to work X hours a week and has Y intensity aspects. I’m less fine with people feeling misled or people feeling unable to communicate about their needs. It seems plausible to me that many of the instances of “Person gets fired or quits and then feels negatively toward Redwood & encourages people not to work there” (which happens, btw) could be avoided/lessened through really good communication/onboarding/expectation-setting. (I have no idea what Redwood’s current procedure is like, but I’d predict that a sharp red-teamer would be able to find 3+ improvements).
These are three examples of interventions that seem valuable and (relatively) low-cost to me. I’d be excited to see if your team came up with any intervention ideas, and I’d be excited to see a “proposed intervention” section in future reports. (Though again, I don’t think you should feel like you need to do this, and I think it’s good to get things out there even if they’re just raising awareness about problems).
We’ve crossposted the full text on LessWrong here: https://www.lesswrong.com/posts/SuZ6Guuos7CjfwRQb/critiques-of-prominent-ai-safety-labs-redwood-research