My guess is that different people do it differently, and I am super weird.
For me a lot of the trick is consciously asking if I am providing good incentives, and remembering to consider what the alternative world looks like.
Zvi
Karma: 40,442
Changes in College Admissions
On Llama-3 and Dwarkesh Patel’s Podcast with Zuckerberg
AI #60: Oh the Humanity
Childhood and Education Roundup #5
Monthly Roundup #17: April 2024
AI #59: Model Updates
I don’t see this response as harsh at all? I see it as engaging in detail with the substance, note the bill is highly thoughtful overall, with a bunch of explicit encouragement, defend a bunch of their specific choices, and I say I am very happy they offered this bill. It seems good and constructive to note where I think they are asking for too much? While noting that the right amount of ‘any given person reacting thinks you went too far in some places’ is definitely not zero.
RTFB: On the New Proposed CAIP AI Bill
Medical Roundup #2
On the 2nd CWT with Jonathan Haidt
AI #58: Stargate AGI
Fertility Roundup #3
Notes on Dwarkesh Patel’s Podcast with Sholto Douglas and Trenton Bricken
AI #57: All the AI News That’s Fit to Print
Economics Roundup #1
On Lex Fridman’s Second Podcast with Altman
Excellent. On the thresholds, got it, sad that I didn’t realize this, and that others didn’t either from what I saw.
I appreciate the ‘long post is long’ problem but I do think you need the warnings to be in all the places someone might see the 10^X numbers in isolation, if you don’t want this to happen, and it probably happens anyway, on the grounds of ‘yes that was technically not a proposal but of course it will be treated like one.’ And there’s some truth in that, and that you want to use examples that are what you would actually pick right now if you had to pick what to actually do (or propose).
I do think the numbers I suggest are about as low as one could realistically get until we get much stronger evidence of impending big problems.
Secrecy is the exception. Mostly no one cares about your startup idea or will remember your hazardous brainstorm, no one is going to cause you trouble, and so on, and honesty is almost always the best policy.
That doesn’t mean always tell everyone everything, but you need to know what you are worried about if you are letting this block you.
On infohazards, I think people were far too worried for far too long. The actual dangerous idea turned out to be that AGI was a dangerous idea, not any specific thing. There are exceptions, but you need a very good reason, and an even better reason if it is an individual you are talking with.
Trust in terms of ‘they won’t steal from me’ or ‘they will do what they promise’ is another question with no easy answers.
If you are planning something radical enough to actually get people’s attention (e.g. breaking laws, using violence, fraud of various kinds, etc) then you would want to be a lot more careful who you tell, but also—don’t do that?
It is better than nothing I suppose but if they are keeping the safeties and restrictions on then it will not teach you whether it is fine to open it up.