I’m working on Impact Markets – markets to trade nonexcludable goods.
If you’re also interested in less directly optimific things – such as climbing around and on top of boulders or amateurish musings on psychology – then you may enjoy some of the posts I don’t cross-post from my blog, Impartial Priorities.
Pronouns: Ideally they. But he/she and gender-neutral neopronouns are fine too.
I didn’t respond to your critiques that went into a more political direction because there was already discussion of those aspects there that I wouldn’t have been able to add anything to. There is concern in the movement in general and in individual EA organizations that because EAs are so predominantly computer scientists and philosophers, there is a great risk of incurring known and unknown unknowns. In the first category, more economists for example would be helpful; in the second category it will be important to bring people from a wide variety of demographics into the movement without compromising its core values. As computer scientist I’m pretty median again.
then there are lots of EA missed opportunities lying around waiting for someone to pick them up
Indeed. I’m not sure if the median EA is concerned about this problem yet, but I wouldn’t be surprised if they are. Many EA organizations are certainly very alert to the problem.
Followed to its logical conclusion, this outlook would result in a lot more concern about the West.
This concern manifests in movement-building (GWWC et al.) and capacity-building (80k Hours, CEA, et al.). There is also concern that I share but that may not yet be median EA concern that we should focus more on movement-wide capacity-building, networking, and some sort of quality over quantity approach to allow the movement to be better and more widely informed. (And by “quantity” I don’t mean to denigrate anyone but just I mean more people like myself who already feel welcomed in the movement because everyone speaks their dialect and whose peers are easily convinced too.)
Throughout the time that I’ve been part of the movement, the general sentiment either in the movement as a whole or within my bubble of it has shifted in some ways. One trend that I’ve perceived is that in the earlier days there was more concern over trying vs. really trying while now concern over putting one’s activism on a long-term sustainable basis has become more important. Again, this may be just my filter bubble. This is encouraging as it shows that everyone is very well capable of updating, but it also indicates that as of one or two years ago, we still had a bunch to learn even concerning rather core issues. In a few more years, I’ll probably be more confident that come core questions are not so much in flux anymore that new EAs can overlook or disregard them and thereby dilute what EA currently stands for or shift it into a direction I couldn’t identify with anymore.
Again, I’m not ignoring your points on political topics, I just don’t feel sufficiently well-informed to comment. I’ve been meaning to read David Roodman’s literature review on open borders–related concerns, since I greatly enjoyed some of his other work, but I haven’t yet. David Roodman now works for the Open Philanthropy Project.
Well, there is a question about what EA is. Is EA about being effectively altruistic within your existing value system? Or is it also about improving your value system to more effectively embody your terminal values? Is it about questioning even your terminal values to make sure they are effective and altruistic?
I’ve always perceived EA as whatever stands at the end of any such process, or maybe not the end but some critical threshold when a person realizes that they agree with the core tenets that they value other’s well-being, and that greater well-being or the well-being or more beings weighs heavier than lesser well-being or the well-being of fewer. If they reach such a threshold. If they do, I see all three processes as relevant.
Regardless of whether you are an antirealist, not all value systems are created equal.
Of course.
Their knowledge of history, politics, and object-level social science is low. … I’m doing the same thing: encouraging EAs to reflect on their value systems, and attain a broader geopolitical and historical context to evaluate their interventions.
Yes, thanks! That’s why I was most interested in your comment in this thread, and because all other comments that piqued my interest in similar ways already had comprehensive replies below them when I found the thread.
This needs to be turned into a concrete strategy, and I’m sure CEA is already on that. Identifying exactly what sorts of expertise are in short supply in the movement and networking among the people who possess just this expertise. I’ve made some minimal-effort attempts to pitch EA to economists, but inviting such people to speak at events like EA Global is surely a much more effective way of drawing them and their insights into the movement. That’s not limited to economists of course.
Do you have ideas for people or professions the movement would benefit from and strategies for drawing them in and making them feel welcome?
I just don’t think a lot of EAs have thought their value systems through very thoroughly
Given how many philosophers there are in the movement, this would surprise me. Is it possible that it’s more the result of the ubiquitous disagreement between philosophers?
How do we know we aren’t also deluded by present-day politics?
I’ve wondered about that in the context of moral progress. Sometimes the idea of moral progress is attacked on the grounds that proponents base their claims for moral progress on how history has developed into the direction of our current status quo, which is rather pointless since by that logic any historical trend toward the status quo would then become “moral progress.” However, by my moral standards the status quo is far from perfect.
Analogously I see that the political views EAs are led to hold are so heterogeneous that some have even thought about coining new terms for this political stance (such as “newtilitarianism”), luckily only in jest. (I’m not objecting to the pun but I’m wary of labels like that.) That these political views are at least somewhat uncommon in their combination suggests to me that we’re not falling into that trap, or at least making an uncommonly good effort of avoiding it. Since the trap is pretty much the default starting point for many of us, it’s likely we still have many legs trapped in it despite this “uncommonly good effort.” The metaphor is already getting awkward, so I’ll just add that some sort of contrarian hypercorrection would of course constitute just another trap. (As it happens, there’s another discussion of the importance of diversity in the context of Open Phil in that Vox article.)
Thanks! I hadn’t seen the formulae for the expected value of perfect information before. I haven’t taken the time to think them through yet, but maybe they’ll come in handy at some point.
Before I delay my reply until I’ve read everything you’ve linked, I’ll rather post a WIP reply.
Thanks for all the data! I hope I’ll have time to look into Open Borders some more in August.
Error theorists would say that the blog post “Effective Altruists are Cute but Wrong” is cute but wrong, but more generally the idea of using PageRank for morality is beautifully elegant (but beautifully elegant things have often turned out imperfect in practice in my experience). I still have to read the rest of the blog post though.
Aw, consoling hugs!
Very good! I’ve incorporated the question, but I’d like to keep the post focused on AGI timelines. :-)
Yeah, but that’s just a network effect where when a critical mass of them start doing it, it becomes normal and they are more at risk of being called stupid for not investing enough.
Good to know!
I just want to send you some sympathy this way. Everything you’ve gone through and all the self-doubt and everything else that I can’t put a name to must be very stressful an exhausting. Reading and responding to hundreds of comments, often very critical ones, is very exhausting too. And who knows what else is going on in your life at the same time. Yet your comments show none of the exhaustion that I would’ve felt in your situation.
I’d also like to second what Rafael already said!
It seems it’s been a few weeks since most of these discussions happened, so I hope you’ve had a chance to relax and recover in the meantime, or I hope that you’ll have some very soon!
Improve your self-esteem and optimism. This is tricky. First, too much self-esteem can lead to harmful narcissism.
A friend of mine uses a great trick here. She has very high self-esteem (though maybe not higher than the esteem that I hold her in) but doesn’t come across narcissistic at all because she’s also very warm and empathetic with a lot of care and curiosity for the people around her. These multiply and are therefore particularly uplifting for those around her: I (for example) not only feel interesting but I feel so interesting that I’m interesting to a person who could find an incredibly interesting person in any random mirror, which, I surmise, must be a very high bar.
So maybe it makes senses to train extraversion, empathy, social skills, and self-esteem all at the same time.
Maybe one should be more hesitant to assign very low probabilities to factors in a conjunction than very high probabilities because the extreme of 100% probability should not be more controversial than omitting a factor.
With a bit of creativity someone could probably come up with dozens of verisimilar additional factor along the lines of “and false vacuum decay hasn’t happened yet.” If we then have to be humble and assign (say) no more than 99% to each of those, it just takes 10 noisy factors to bias the estimate down to ~ 90%.
In this case I went with Nate’s approach and merged the last three factors. None of them seemed silly to me, but I felt like splitting them up didn’t help my intuitions.
That’s a great report and exercise! Thank you!
I hopelessly anchored on almost everything in the report, so my estimate is far from independent. I followed roughly Nate’s approach (though that happened without anchoring afaik), and my final probability is ~ 50% (+/- 5% when I play around with the factors). But it might’ve turned out differently if I had had an espresso more or less in the morning.
50% is lower than what I expected – another AI winter would lead to a delay and might invalidate the scaling hypothesis, so that the cumulative probability should probably increase more slowly after 2040–50, but I would’ve still expected something like 65–85%.
My biggest crux seems to be about the necessity for “deployment.” The report seems to assume that for a system to become dangerous, someone has to decide to deploy it. I don’t know what the technology will be like decades from now, but today I test software systems hundreds of times on my laptop and on different test systems before I deploy them. I’m the only user of my laptop, so privilege escalation is intentionally trivial. As it happens it’s also more powerful than the production server. Besides, my SSH public key is in authorized_keys files on various servers.
So before I test a software, I likely will not know whether it’s aligned, and once I test it, it’s too late. Add to that that even if many people should indeed end up using carefully sandboxed systems for testing AGIs within a few decades, it still only takes one person to relax their security a bit.
So that’s why I merged the last three points and collectively assigned ~ 95% to them (not anchored on Nate afaik). (Or to be precise I mostly ignored the last point because it seems like a huge intellectual can of beans spanning ethics, game theory, decision theory, physics, etc., so more than the purview of the report.)
Thanks for the analysis!
Some of these points seem like attempts at being conservative (I imagine most people will prefer to err in the direction of caution when advising others), while other points just look like the project is not well-maintained anymore.
Do you have a feeling for what a good adjustment factor is that one could apply to the result to compensate the first kind of problem, so everything related to others’ vaccination status? (I’ll just enter the local data manually while the importer is broken.)
(I’m hesitant to go with a budget that would imply a 50% yearly risk for me, but if I’m going to stick with my approximate 3% p.a. risk budget, I want to do so using a median estimate, not a conservative one. Update: Did some math. I think I’m comfortable with a 20% p.a. budget now. But I don’t really do budgets because I weigh for every event what risk it is worth or whether there’s an equally good but safer alternative.)
Good. There’s a site for Switzerland like that too. I extrapolated from that in a similar manner. :-)
My friend Rai said in a Telegram chat:
So my thinking is something like: If you just throw money at FOSS ML libraries, I expect you’d mostly shorten the time from “some researcher writes a paper” and “that model is used in a billion real-world projects”. I think you’d make whatever AI tech exists be more broadly distributed. I don’t think that would directly make stronger AI arrive faster, because I think it would mostly just give people lots of easy-to-use boxes, like when CNNs got popularized, it became quite trivial to train any sort of visual classification task you wanted.
But I can see that this would make weaker AI get deployed faster, so that gains from capabilities research could be easier to see for investors, so it could indirectly pour more effort into developing stronger capabilities.
The change of timelines that would produce would depend on to what degree you believe that we are in a “deployment overhang”, in the sense of “practical deployment of models is lagging far behind what we have capacities for”. I can see arguments for both why we are and aren’t in such an overhang. Argument for would be of the shape “have you seen the stuff GPT-3 can do? Why doesn’t everyone have a competent virtual assistant yet?” An argument against would be “look at how slow we’re going with e.g. self-driving cars or AI in medicine, capabilities research just glosses over lots of stuff that blocks practical deployment”.
I think it also depends a lot on what exactly you’d be funding. For example, “cutting-edge pre-trained deep RL for everyone” versus “explainable interpretable robust models for everyone”
Me:
Woot, thanks! Can I share that with the others, including the author of the question? My interpretation was even that it’s about libraries like cchardet – super fundamental ones. Lots of websites misdeclare their encodings, so you get a lot of messy text with encoding errors when you try to read it. Make cchardet a bit more magical, add support for a few more encodings, and you can extract a bit more training data, or a developer saves a day of hard-coding the right encodings to use for a bunch of domains. That, except with thousands and thousands of libraries like cchardet.
Rai:
Sure you can share that, even as a comment with attribution to me if you think it could be useful.
With stuff like more fundamental libraries there it becomes i think a question of generally speeding up industry/innovation. I think you’d probably want to plug all this into a general development model.
Hi! I suppose that question mostly goes to Adam? The importer is fixed, so I’m not doing any of this anymore. What I did was to extrapolate to the current, incomplete week n using the slope from week n-2 to n-1. Then I set the percent increase to 0 because the week is actually the current week already.
But the differences from week to week will, in most cases, be minor, so I don’t think it’s important to get this exactly right. There are so many other uncertain factors that go into the model that I don’t recommend investing too much time into this one.
Agreed. Also here’s the poem that goes with that comment:
Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.Though wise men at their end know dark is right,
Because their words had forked no lightning they
Do not go gentle into that good night.Good men, the last wave by, crying how bright
Their frail deeds might have danced in a green bay,
Rage, rage against the dying of the light.Wild men who caught and sang the sun in flight,
And learn, too late, they grieved it on its way,
Do not go gentle into that good night.Grave men, near death, who see with blinding sight
Blind eyes could blaze like meteors and be gay,
Rage, rage against the dying of the light.And you, my father, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.
I totally empathize with Eliezer, and I’m afraid that I might be similarly burned out if I had been trying this for as long.
But that’s not who I want to be. I want to be Harry who builds a rocket to escape Azkaban, the little girl that faces the meteor with a baseball bat, and the general who empties his guns into the sky against another meteor (minus all his racism and shit).
I bet I won’t always have the strength for that – but that’s the goal.
As someone said in another comment there are the core tenets of EA, and there is your median EA. Since you only seem to have quibbles with the latter, I’ll address some of those, but I don’t feel like accepting or rejecting them is particularly important for being an EA in the context of the current form of the movement. We love discussing and challenging our views. Then again I think I so happen to agree with many median EA views.
VoiceOfRa put very concisely what I think is a median EA view here, but the comment is so deeply nested that I’m afraid it might get buried: “Even if he values human lives terminally, a utilitarian should assign unequal instrumental value to different human lives and make decision based on the combination of both.”
I think this has been mentioned in the comments but not very directly. The median EA view may be not to bother with philosophy at all because the branches that still call themselves philosophy haven’t managed to come to a consensus on central issues over centuries so that there is little hope for the individual EA to achieve that.
However when I talk to EAs who do have a background in philosophy, I find that a lot of them are metaethical antirealists. Lukas Gloor, who also posted in this thread, has recently convinced me that antirealism, though admittedly unintuitive to me, is the more parsimonious view and thus the view under which I operate now. Under antirealism moral intuitions, or some core ones anyway, are all we have, so that there can be no philosophical arguments (and thus no good or bad ones) for them.
Even if this is not a median EA view, I would argue that most EAs act in accordance with it just out of concern for the cost-effectiveness of their movement-building work. It is not cost-effective to try to convince everyone of the most unintuitive inferences from ones own moral system. However, among the things that are important to the individual EA, there are likely many that are very uncontroversial in most of society and focusing on those views in one’s “evangelical” EA work is much more cost-effective.
From my moral vantage point, the alternative (I’ll consider a different counterfactual in a moment) that I keep the money to spend it on myself where its marginal positive impact on my happiness is easily two or three orders of magnitude lower and my uncertainty over what will make me happy is also just slightly lower than with some top charities, that alternative would be a much more extraordinary claim.
You could break that up and note that in the end I’m not deciding to just “donate effectively,” but that I’ll decide on a very specific intervention and charity to donate to, for example Animal Equality, making my decision much more shaky again, but I’d also have to make such highly specific decisions that are probably only slightly less shaky when trying to spend money on my own happiness.
However, the alternative might also be:
That’s something the median EA has probably considered a good deal. Even at GiveWell there was a time in 2013 when some of the staff pondered whether it would be better to hold off with their personal donations and donate a year later when they’ve discovered better giving opportunities.
However several of your arguments seem to stem from uncertainty in the sense of “There is substantial uncertainty, so we should hold off doing X until the uncertainty is reduced.” Trading off these element in an expected value framework and choosing the right counterfactuals is probably again a rather personal decision when it comes to investing ones donation budget, but over time I’ve become less risk-averse and more ready to act under some uncertainty, which has hopefully brought me closer to maximizing the expected utility of my actions. Plus I don’t expect any significant decreases in uncertainty wrt the best giving opportunities in the future that I could wait for. There will hopefully be more with similar or only slightly greater levels of uncertainty though.