Super excited that someone has made an online version of the game, here: https://aumann.io/
Davidmanheim
Karma: 3,355
You don’t change a law, you pass legislation that does something specific. And that legislation can carve out whatever the person writing it says it should—not just repealing the Dredge Act, but also slightly modifying the Jones Act to limit its scope. And that’s exactly what the mentioned solutions do, if you read them.
Question: “effective arguments for the importance of AI safety”—is this about arguments for the importance of just technical AI safety, or more general AI safety, to include governance and similar things?
Wait, the goal here, at least, isn’t to produce truth, it is to disseminate it. Counter-arguments are great, but this isn’t about debating the question, it’s about communicating a conclusion well.
Think of it as a “practicing a dark art of rationality” post, and I’d think it would seem less off-putting.
Nope, find me online, I’m pretty easy to reach.
If you’re interested, I’d start here: https://www.alignmentforum.org/posts/YAa4qcMyoucRS2Ykr/basic-inframeasure-theory and go through the sequence. (If you’re not comfortable enough with the math involved, start here first: https://www.lesswrong.com/posts/AttkaMkEGeMiaQnYJ/discuss-how-to-learn-math )
And if you’ve gone through the sequence and understand it, I’d suggest helping developing the problem sets that are mentioned in one of the posts, or reaching out to me.
For this position, we are looking for people already able to contribute at a very high level. If you’re interested in working on the agenda to see if you’d be able to do this in the future, I’d be interested in chatting separately and looking at whether some form of financial support or upskilling would be useful, and look at where to apply for funding.
Please feel free to repost this elsewhere, and/or tell people about it.
And if there is anyone interested in this type of job, but is currently still in school or for other reasons is unable to work full time at present, we encourage them to apply and note the circumstances, as we may be able to find other ways to support their work, or at least collaborate and provide mentorship.
The initial section seems very closely related to the work Scott did and we wrote up in the “Three Causal Goodhart Effects” of this paper; https://arxiv.org/pdf/1803.04585.pdf, including some of the causal graphs.
Regarding mitigations, see my preprint, here: https://mpra.ub.uni-muenchen.de/98288/, which in addition to some of the mitigations you discussed, also suggest secret metrics, randomization, and post-hoc specification as strategies. Clearly, these don’t always apply in AI systems, but can potentially be useful in at least some cases.
Agreed that the proposal combines knowledge with calibration, but your procedure doesn’t actually seem implementable.
There is the brier score, or any other proper scoring rule. These each have the advantage of being zero-degree-of-freedom up to the choice of scoring rule, though it isn’t information preserving, and isn’t comparable across different sets of predictions. (Though neither is any analogue of a CDF.)
This was pointed out by someone working in ML—but I can’t find major conference deadlines that this would have been in response to, so maybe it’s not a useful explanation.
But NeurIPS submissions open this week—these may have been intended for that, or precede that to claim priority in case someone else submits something similar. I’d be interested in looking at release dates / times for previous big advances compared to conferences.
Announcements of progress tend to clump together before the major AI conferences.
Just noting that I think you’re arguing strongly against what is at most a weak man argument. (And given that the author retracted the post, it might just be a straw-man.)
Super excited to see the offers to bet, though.
Fine, but the offer was for “up to $1000”
Wanted to note that “proof of work” seems to correspond to “skin in the game,” which I’d be willing to bet NN Taleb would claim is an ancient philosophical concept and justification for knowledge.
Kind of—see my response quoting the book, above.
“There is significant uncertainty remaining in these estimates and they should be treated as representing the right order of magnitude—each could easily be a factor of 3 higher or lower.”
“In the case of artificial intelligence, everyone agrees the evidence and arguments are far from watertight, but the question is where does this leave us? Very roughly, my approach is to start with the overall view of the expert community that there is something like a one in two chance that AI agents capable of outperforming humans in almost every task will be developed in the coming century.”
″Some of my colleagues give higher chances than me, and some lower. But for many purposes our numbers are similar. Suppose you were more skeptical of the risk and thought it to be one in 100. From an informational perspective, that is actually not so far apart: it doesn’t take all that much evidence to shift someone from one to the other. And it might not be that far apart in terms of practical action either—an existential risk of either probability would be a key global priority.”
- Toby Ord, The Precipice
This was absolutely correct in the past, and is the reason that WTO anti-dumping regulations exist, and as of 1995, are pretty well managed and enforceable. https://www.wto.org/english/tratop_e/adp_e/adp_info_e.htm