Yes please
tamgent
Karma: 84
Do you have the transcript from this?
I like it—interesting how much is to do with the specific vulnerabilities of humans, and how humans exploiting other humans’ vulnerabilities was what enabled and exacerbated the situation.
There’s also a romantic theme ;-)
Whilst we’re sharing stories...I’ll shamelessly promote one of my (very) short stories on human manipulation by AI. In this case the AI is being deliberative at least in achieving its instrumental goals. https://docs.google.com/document/d/1Z1laGUEci9rf_aaDjQKS_IIOAn6D0VtAOZMSqZQlqVM/edit
Is it a coincidence that your handle is blaked? (It’s a little similar to Blake) Just curious.
Ha! I meant the former, but I like your second interpretation too!
I like, ‘do the impossible—listen’.
Recruitment—in my experience often a weeks long process from start to finish, well oiled and systematic and using all the tips from the handbook on organizational behaviour on selection, often with feedback given too. By comparison, some tech companies can take several months to hire, with lots of ad hoc decision-making, no processes around biases or conflicts of interest, and no feedback.
Happy to give more examples if you want by DM.
I should say my sample size is tiny here—I know one gov dept in depth, one tech company in depth and a handful of other tech companies and gov depts not fully from the inside but just from talking with friends that work there, etc.
What exactly is the trust problem you’re referring to?
Is it you think that people are not as trusting as you think they should be, in general?
I also interpreted it this way and was confused for a while. I think your suggested title is clearer, Neel.
Thank you for writing this. On your section ‘Obstruction doesn’t need discernment’ - see also this post that went up on LW a while back called The Regulatory Option: A response to near 0% survival odds. I thought it was an excellent post, and it didn’t get anywhere near the attention it deserved, in my view.
I think the two camps are less orthogonal than your examples of privacy and compute reg portray. There’s room for plenty of excellent policy interventions that both camps could work together to support. For instance, increasing regulatory requirements for transparency on algorithmic decision-making (and crucially, building a capacity both in regulators and in the market supporting them to enforce this) is something that I think both camps would get behind (the xrisk one because it creates demand for interpretability and more and the other because eg. it’s easier to show fairness issues) and could productively work on together. I think there are subculture clash reasons the two camps don’t always get on, but that these can be overcome, particularly given there’s a common enemy (misaligned powerful AI). See also this paper Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society I know lots of people who are uncertain about how big the risks are, and care about both problems, and work on both (I am one of these—I care more about AGI risk, but I think the best things I can do to help avert it involve working with the people you think aren’t helpful).
To build on the benefit you noted here:
better citability (e.g. if somebody writes an ML paper to be published in ML venues, it gives more credibility to cite arXiv papers than Alignment Forum/LessWrong posts.
There are some areas of work whereby it’s useful to not be implicitly communicating that you affiliate with a somewhat weird group like LW or AF folks but you want the content to be read at face value when you share it with folks who are coming from different subcultures and perspectives. I think it’d be hugely valuable for this collection of people who are sharing things.
This seems solvable and very much worth solving!
Agree.
Human values are very complex and most recommender systems don’t even try to model them. Instead most of them optimise for things like ‘engagement’ which they claim to be aligned with a user’s ‘revealed preference’. This notion of ‘revealed preference’ is a far cry from true preferences (which are very complex) let alone human values (which are also very complex). I recommend this article for an introduction to some of the issues here: https://medium.com/understanding-recommenders/what-does-it-mean-to-give-someone-what-they-want-the-nature-of-preferences-in-recommender-systems-82b5a1559157
Support.
I would add to this that The Alignment Problem by Brian Christian is a fantastic general audience book that shows how the immediate and long-term AI policy really are facing the same problem and will work better if we all work together.
If you know of any more such analyses could you share?
I would be interested in seeing a list of any existing work in this area. I think determining the red lines well are going to be very useful for policymakers in the next few years.
Not a textbook (more for a general audience) but The Alignment Problem by Brian Christian is a pretty good introduction that I reckon most people interested in this would get behind.