chanamessinger
which privacy skills you are able to execute.
This link goes to a private google doc, just fyi.
This is great!
I really like this about slack:
If you aren’t maintaining this, err on the side of cultivating this rather than doing high-risk / high-reward investments that might leave you emotionally or financially screwed.
(or, if you do those things, be aware I may not help you if it fails. I am much more excited about helping people that don’t go out of their way to create crises)
Seems like a good norm and piece of advice.
I’m confused how much I should care whether an impact assessment is commissioned by some organization. The main thing I generally look for is whether the assessment / investigation is independent. The argument is that because AISC is paying for it, that will influence the assessors?
I have not read most of what there is to read here, just jumping in on “illegal drugs” ---> ADHD meds. Chloe’s comment spoke to weed as the illegal drug on her mind.
AI has immense potential, but also immense risks. AI might be misused by China, or get of control. We should balance the needs for innovation and safety.” I wouldn’t call this lying (though I agree it can have misleading effects, see Issue 1).
Not sure where this slots in, but there’s also a sense in which this contains a missing positive mood about how unbelievably good (aligned) AI could or will be, and how much we’re losing by not having it earlier.
Thanks!
Interesting how many of these are “democracy / citizenry-involvement” oriented. Strongly agree with 18 (whistleblower protection) and 38 (simulate cyber attacks).
20 (good internal culture), 27 (technical AI people on boards) and 29 (three lines of defense) sound good to me, I’m excited about 31 if mandatory interpretability standards exist.
42 (on sentience) seems pretty important but I don’t know what it would mean.
The top 6 of the ones in the paper (the ones I think got >90% somewhat or strongly agree, listed below), seem pretty similar to me—are there important reasons people might support one over another?
Pre-deployment risk assessments
Evaluations of dangerous capabilities
Third-party model audits
Red teaming
Pre-training risk assessments
Pausing training of dangerous models
Curious if you have any updates!
Chat GPT gives some interesting analysis when asked, though I think not amazingly accurate. (The sentence I gave it, from here is a weird example, though)
Does it say anything about AI risk that is about the real risks? (Have not clicked the links, the text above did not indicate to me one way or another).
This is great, and speaks to my experience as well. I have my own frames that map onto some of this but don’t hit some of the things you’ve hit and vice versa. Thanks for writing!
Is this something Stampy would want to help with?
https://www.lesswrong.com/posts/WXvt8bxYnwBYpy9oT/the-main-sources-of-ai-risk
I think that incentivizes self-deception on probabilities. Also, P <10^-10 are pretty unusual, so I’d expect that to cause very little to happen.
Thanks!
When you say “They do, however, have the potential to form simulacra that are themselves optimizers, such as GPT modelling humans (with pretty low fidelity right now) when making predictions”
do you mean things like “write like Ernest Hemingway”?
Is it true that current image systems like stable diffusion are non-optimizers? How should that change our reasoning about how likely it is that systems become optimizers? How much of a crux is “optimizeriness” for people?
Why do people keep saying we should maximize log(odds) instead of odds? Isn’t each 1% of survival equally valuable?
In addition to Daniel’s point, I think an important piece is probabilistic thinking—the AGI will execute not based on what will happen but on what it expects to happen. What probability is acceptable? If none, it should do nothing.
Have you written about your update to slow takeoff?
I feel pretty sympathetic to the desire not to do things by text; I suspect you get much more practiced and checked over answers that way.