I think establishing whether AI has intent in the legal sense will probably be a mess. Strict liability is probably better because it averts that, but it should be tied only to the highest-risk activities. I like Weil’s proposal for categorizing certain deployments of frontier AI as abnormally dangerous activities, which would let us use strict liability.
sanyer
Karma: 280
and also to plausibly win, which seems less good to aim for here
I don’t think it would be wise or feasible for middle powers to try to ‘win’ the AI race, nor do I think this is a threat in any meaningful way.
The only ‘winning’ here that is feasible for middle powers is that both safety is ensured and benefits of AGI are shared fairly. Those are important goals in their own rights and seem like things we should aim for.
Better forecasting infrastructure will help the Dems allocate resources in the 2028 election. In fact, it likely helped the Dems keep the House close in 2024, which has provided an important check on Republican power over the last year or two. Betting markets have outperformed polling aggregators like 538 or the NYT since they took off in popularity and will continue to do so. This will help Democrats allocate funding to tipping point congressional races and is probably worth millions of dollars alone, if not far more (see recent EA focus on democracy).
This seems to imply that better forecasting infrastructure helps Democrats more than Republicans? Why would we expect that to be the case? To me, it seems like prediction markets are more popular on the right so I would expect Republicans to benefit more from forecasting infrastructure.
Prediction markets, or at least judgemental forecasting, has been used in companies before though? At least Google had a pretty substantive internal forecasting platform at some point.
Is there something specifically about EA groups that make them more favorable to an AI pause compared to national AI safety groups? AI safety groups seem like potentially more credible actors regarding this (and indeed, at least CeSIA in France has already done stuff regarding this)
What is the ideology in middle powers you fear of? What are the harmful actions that middle powers might do if they get AGI-pilled?
(Maybe you might be worried that these countries attempt building their own AGI, but it seems extremely unlikely these countries would be able to race ahead the US or China. The worst that could come from that is slightly worse arms race and lots of middle power money wasted)
I guess one consideration for prioritizing direct work over capacity-building is AI timelines. Do you think this should be an important consideration?
[Question] When has forecasting been useful for you?
Is there any particular reason to expect they would be memetic viruses?
They show up to me now as well. Not sure what happened yesterday, weird
Not sure what happened in between, but some diagrams are missing again
The soundness of this advice depends a bit on what career path you want to pursue, though. If you want to do some lobbying or policy advocacy, it’s pretty difficult to “just get to work” if you don’t have the right network, skill set, and credentials. And working in that area without knowing what you’re doing can also be quite harmful.
It’s worth saying that applying for things can also yield some benefits. I definitely became a better writer through the various work tests I did when I was applying to lots of training programs. I also got some nice feedback (props to CLR especially!) and the experience helped me to better understand what different orgs & people are working on. I also got a clearer idea of my career aspirations.
This is assuming you get through the very first round and get to do some test tasks though...
Well yes, but I’m not sure if non-native speakers are in the “intended public”, since they operate in the US mostly
As a non-native English speaker, OpenPhil was sooo much easier to pronounce than Coefficient Giving. I’m sure this shouldn’t play a big part in the naming decision, but still...
What are some examples of illegible AI safety problems?
Regarding results of empirical evaluations of AI’s scheming-relevant capabilities, I think we could do even better than simple removal by replacing the real results with synthetic results that intentionally mislead the AIs about their own capabilities. So, if the model is good at scheming, we could mislead it by making it believe it is bad at it, and vice-versa. I think this could be quite feasible, since rather than coming up with fully synthetic data that may be easy to spot as synthetic, you only need to replace some specific results.
A link to the original article would be appreciated
How many people are working on test-time learning? How feasible do you think it is?
To what extent do the Singapore AI Safety Priorities capture what you care about?