AI engineer
alyssavance
Karma: 3,470
Is AI Progress Impossible To Predict?
Humans are very reliable agents
Visible Homelessness in SF: A Quick Breakdown of Causes
Almost everyone should be less afraid of lawsuits
Levels of Action
Negative and Positive Selection
Failing to fix a dangerous intersection
Science: Do It Yourself
AI Training Should Allow Opt-Out
Vote up this comment if you would be most likely to read a post on Less Wrong or another friendly blog.
I appreciate the effort, and I agree with most of the points made, but I think resurrect-LW projects are probably doomed unless we can get a proactive, responsive admin/moderation team. Nick Tarleton talked about this a bit last year:
“A tangential note on third-party technical contributions to LW (if that’s a thing you care about): the uncertainty about whether changes will be accepted, uncertainty about and lack of visibility into how that decision is made or even who makes it, and lack of a known process for making pull requests or getting feedback on ideas are incredibly anti-motivating.” (http://lesswrong.com/lw/n0l/lesswrong_20/cy8e)
That’s obviously problematic, but I think it goes way beyond just contributing code. As far as I know, right now, there’s no one person with both the technical and moral authority to:
set the rules that all participants have to abide by, and enforce them
decide principles for what’s on-topic and what’s off-topic
receive reports of trolls, and warn or ban them
respond to complaints about the site not working well
decide what the site features should be, and implement the high-priority ones
Pretty much any successful subreddit, even smallish ones, will have a team of admins who handle this stuff, and who can be trusted to look at things that pop up within a day or so (at least collectively). The highest intellectual-quality subreddit I know of, /r/AskHistorians, has extremely active and rigorous moderation, to the extent that a majority of comments are often deleted. Since we aren’t on Reddit itself, I don’t think we need to go quite that far, but there has to be something in place.
Why Academic Papers Are A Terrible Discussion Forum
AlphaGeometry: An Olympiad-level AI system for geometry
A Quick Note on AI Scaling Asymptotes
I think saying “we” here dramatically over-indexes on personal observation. I’d bet that most overweight Americans have not only eaten untasty food for an extended period (say, longer than a month); and those that have, found that it sucked and stopped doing it. Only eating untasty food really sucks! For comparison, everyone knows that smoking is awful for your health, it’s expensive, leaves bad odors, and so on. And I’d bet that most smokers would find “never smoke again” easier and more pleasant (in the long run) than “never eat tasty food again”. Yet, the vast majority of smokers continue smoking:
https://news.gallup.com/poll/156833/one-five-adults-smoke-tied-time-low.aspx
I edited the MNIST bit to clarify, but a big point here is that there are tasks where 99.9% is “pretty much 100%” and tasks where it’s really really not (eg. operating heavy machinery); and right now, most models, datasets, systems and evaluation metrics are designed around the first scenario, rather than the second.
Intentional murder seems analogous to misalignment, not error. If you count random suicides as bugs, you get a big numerator but an even bigger denominator; the overall US suicide rate is ~1:7,000 per year, and that includes lots of people who have awful chronic health problems. If you assume a 1:20,000 random suicide rate and that 40% of people can kill themselves in a minute (roughly, the US gun ownership rate), then the rate of not doing it per decision is ~20,000 * 60 * 16 * 365 * 0.4 = 1:3,000,000,000, or ~99.99999997%.
You say “yet again”, but random pilot suicides are incredibly rare! Wikipedia counts eight on commercial flights in the last fifty years, out of a billion or so total flights, and some of those cases are ambiguous and it’s not clear what happened: https://en.wikipedia.org/wiki/Suicide_by_pilot
This is just a guess, but I think CFAR and the CFAR-sphere would be more effective if they focused more on hypothesis generation (or “imagination”, although that term is very broad). Eg., a year or so ago, a friend of mine in the Thiel-sphere proposed starting a new country by hauling nuclear power plants to Antarctica, and then just putting heaters on the ground to melt all the ice. As it happens, I think this is a stupid idea (hot air rises, so the newly heated air would just blow away, pulling in more cold air from the surroundings). But it is an idea, and the same person came up with (and implemented) a profitable business plan six months or so later. I can imagine HPJEV coming up with that idea, or Elon Musk, or von Neumann, or Google X; I don’t think most people in the CFAR-sphere would, it’s just not the kind of thing I think they’ve focused on practicing.
Arrow’s Theorem is a Lie
Clients are free to publish whatever they like, but we are very strict about patient confidentiality, and do not release any patient information without express written consent.
Fantastic post! I agree with most of it, but I notice that Eliezer’s post has a strong tone of “this is really actually important, the modal scenario is that we literally all die, people aren’t taking this seriously and I need more help”. More measured or academic writing, even when it agrees in principle, doesn’t have the same tone or feeling of urgency. This has good effects (shaking people awake) and bad effects (panic/despair), but it’s a critical difference and my guess is the effects are net positive right now.