I think saying “we” here dramatically over-indexes on personal observation. I’d bet that most overweight Americans have not only eaten untasty food for an extended period (say, longer than a month); and those that have, found that it sucked and stopped doing it. Only eating untasty food really sucks! For comparison, everyone knows that smoking is awful for your health, it’s expensive, leaves bad odors, and so on. And I’d bet that most smokers would find “never smoke again” easier and more pleasant (in the long run) than “never eat tasty food again”. Yet, the vast majority of smokers continue smoking:https://news.gallup.com/poll/156833/one-five-adults-smoke-tied-time-low.aspx
https://transformer-circuits.pub/ seems impressive to me!
There are now quite a lot of AI alignment research organizations, of widely varying quality. I’d name the two leading ones right now as Redwood and Anthropic, not MIRI (which is in something of a rut technically). Here’s a big review of the different orgs by Larks:
Great post. I’m reminded of instructions from the 1944 CIA (OSS) sabotage manual:”When possible, refer all matters to committees, for “further study and consideration.” Attempt to make the committee as large as possible — never less than five.”
Eliezer’s writeup on corrigibility has now been published (the posts below by “Iarwain”, embedded within his new story Mad Investor Chaos). Although, you might not want to look at it if you’re still writing your own version and don’t want to be anchored by his ideas.
Would be curious to hear more about what kinds of discussion you think are net negative—clearly some types of discussion between some people are positive.
Thanks for writing this! I think it’s a great list; it’s orthogonal to some other lists, which I think also have important stuff this doesn’t include, but in this case orthogonality is super valuable because that way you’re less likely for all lists to miss something.
This is an awesome comment, I think it would be great to make it a top-level post. There’s a Facebook group called “Information Security in Effective Altruism” that might also be interested
I hadn’t seen that, great paper!
Fantastic post! I agree with most of it, but I notice that Eliezer’s post has a strong tone of “this is really actually important, the modal scenario is that we literally all die, people aren’t taking this seriously and I need more help”. More measured or academic writing, even when it agrees in principle, doesn’t have the same tone or feeling of urgency. This has good effects (shaking people awake) and bad effects (panic/despair), but it’s a critical difference and my guess is the effects are net positive right now.
I edited the MNIST bit to clarify, but a big point here is that there are tasks where 99.9% is “pretty much 100%” and tasks where it’s really really not (eg. operating heavy machinery); and right now, most models, datasets, systems and evaluation metrics are designed around the first scenario, rather than the second.
Intentional murder seems analogous to misalignment, not error. If you count random suicides as bugs, you get a big numerator but an even bigger denominator; the overall US suicide rate is ~1:7,000 per year, and that includes lots of people who have awful chronic health problems. If you assume a 1:20,000 random suicide rate and that 40% of people can kill themselves in a minute (roughly, the US gun ownership rate), then the rate of not doing it per decision is ~20,000 * 60 * 16 * 365 * 0.4 = 1:3,000,000,000, or ~99.99999997%.
You say “yet again”, but random pilot suicides are incredibly rare! Wikipedia counts eight on commercial flights in the last fifty years, out of a billion or so total flights, and some of those cases are ambiguous and it’s not clear what happened: https://en.wikipedia.org/wiki/Suicide_by_pilot
Crazy idea: LessWrong and EA have been really successful in forming student groups at elite universities. But in the US, elite university admissions select on some cool traits (eg. IQ, conscientiousness), don’t select on others, and anti-select on some (eg. selection against non-conformists). To find capable people who didn’t get into an elite school, what if someone offered moderate cash bounties (say, $1,000-$5,000 range) to anyone who could solve some hard problem (eg. an IMO gold medal problem, or something like https://microcorruption.com/), without already being a “recognized expert” (say, under age 25, not at an elite school, not already working in the field, etc.). This would be similar to the old “Quixey Challenge” (https://venturebeat.com/2012/06/09/quixey-gamifies-job-hunting-in-an-entirely-new-way/), where coders were offered $100 to find a bug in one minute, but at a slightly broader scale and for different skills. This would select from a broader range of people, and could be adapted to whatever types of skills are most in demand, eg. hacking challenges for trying to recruit computer security people (https://forum.effectivealtruism.org/posts/ZJiCfwTy5dC4CoxqA/information-security-careers-for-gcr-reduction).
I think it’s true, and really important, that the salience of AI risk will increase as the technology advances. People will take it more seriously, which they haven’t before; I see that all the time in random personal conversations. But being more concerned about a problem doesn’t imply the ability to solve it. It won’t increase your base intelligence stats, or suddenly give your group new abilities or plans that it didn’t have last month. I’ll elide the details because it’s a political debate, but just last week, I saw a study that whenever one problem got lots of media attention, the “solutions” people tried wound up making the problem worse the next year. High salience is an important tool, but nowhere near sufficient, and can even be outright counterproductive.
On the state level, the correlation between urbanization and homelessness is small (R^2 = 0.13) and disappears to zero when you control for housing costs, while the reverse is not true (R^2 of the residual = 0.56). States like New Jersey, Rhode Island, Maryland, Illinois, Florida, Connecticut, Texas, and Pennsylvania are among the most urbanized but have relatively low homelessness rates, while Alaska, Vermont, and Maine have higher homelessness despite being very rural. There’s also, like, an obvious mechanism where expensive housing causes homelessness (poor people can’t afford rent).
The correlation between homelessness and black population in a state is actually slightly negative (R^2 = 0.09). Mississippi, Louisiana and Alabama are some of the blackest states and have the lowest homelessness rates in the US; Hawaii, Oregon, Alaska and Vermont have some of the highest despite being <5% black.
Data from 538′s Urbanization Index: https://fivethirtyeight.com/features/how-urban-or-rural-is-your-state-and-what-does-that-mean-for-the-2020-election/
Do you have links handy?
Thanks, I hadn’t seen that! Added it to the post