Running https://aiplans.org
Fulltime working on the alignment problem.
Kabir Kumar
Karma: 914
Hi!! I’m working on a course and other projects on this, to help US lawyers! Held an AI Safety Law-a-Thon in October to get more people informed and working on this, multiple lawyers, including one law firm owner in the US said it changed their careers a lot—would like to work together on this!!
easy thing to do: commit to voting against ai datacenters and convince at least one other person to also do so—on the argument of extinction
extra: commit to voting against ai datacenters and convince at least one other person to also do so and convince them such that they also want to convince at least one other person—on the argument of extinction
Here is a Goldman Sachs report: https://www.ansa.it/documents/1680080409454_ert.pdf
They make essentially 2 predictions on the first page and the rest of the 20 pages, is explaining their reasoning.
Compare this to AI 2027, just what they say for Early 2026: “Several competing publicly released AIs now match or exceed Agent-0, including an open-weights model. OpenBrain responds by releasing Agent-1, which is more capable and reliable.”—and what appears when you hover over what might appear to be a citation: “In practice, we expect OpenBrain to release models on a faster cadence than 8 months, but we refrain from describing all incremental releases for brevity.”Why do they expect this? What are their sources? In what ways could they be biased? How do we know if their sources become outdated, or not as reliable?
We aren’t given this information.
Compare this to the prediction in Goldman Sachs: “The boost to global labor productivity could also be economically significant, and we estimate that AI could eventually increase annual global GDP by 7%.”
We’re given a specific source and specific data.
And what’s under their number? Is it another claim whose source we don’t know? No, it’s a specific source that we can look up and use to get more knowledge—and potentially use to make counterarguments to the prediction and analysis by Goldman Sachs, by either calling into question the value of the source, the interpretation of the data, or a number of other things.
Comparatively, how can we actually make counterarguments against AI 2027? How can we tell if their sources are high quality, or have been interpreted in ways that we would agree with?
I think AI 2027 makes much much more of an Argument From Authority, which I consider to be a reduction in the quality of discourse.
this is great and more of this is needed, thank you
i think forecasting is kinda like Analysts, by analysts also shared their *reasoning* - pros of forecssters not doing this is they get to use vibes more, use weird kinda wrong but directionally correct vibes more, etc but cons are they become more monopolistic on the reasoning
and we get more concentration of power type stuff
and the reasoning is actually often by far the most useful part of an analysis
yes, this makes sense to me
I predict Fable 5 will have this self bias typed of behaviour across multiple languages, multiple personas, etc
mostly the former
it kept doubling down on insisting that a quote that included “5 day version” was something I put into the message. Imo, fable is a capable enough model to know what is going on here. This, combined with the recent paper of models being biased towards their own orgs/companies makes me think Fable has a good chance—a chance, to be clear—of being consistent in it’s lack of full honesty when talking about itself.
Yes, I call this deceptive. I’m switching to Sol due to it.
i think it would be good for you to just try this, have some way for tracking it (doesnt need to be super precise, quick and actually done is better than super elaborate and not done here, imo), and see what happens!! i’m also curious!
awesome!!
Fund AFFINE
Fable hallucinates and continues doubling down even after 3 messages to the contrary. It hallucinated the line “check gpt 5.6′s suggestions for a 5 day version and consider whether the two of you disagree anywhere” and kept doubling down even when contradicting evidence was shown.
yes, I agree, was thinking of this last night as the thing that lesswrong has atm to go against this bias
also, lesswrong is largely a place for entertainment, posts that aren’t offensive but are entertaining and make the reader feel smarter for having read it, get more interaction and upvotes than things that are actually helpful/useful, etc—and because its browsed by neel nanda, richard ngo, etc type, it’ll get seen by them and as long as its not too repulsive to them, just that its getting this high number and lots of interactions and also is entertaining, will mean that it has a higher chance of getting read and then promoted and other stuff.
Also, the best thing for lesswrong posting is to have something in the post that *could* and is *likely* to be misinterpreted and called out by someone who isnt super bright, but is confident, but be able to slam them down decisively and show that you were clearly correct all along and look really smart—esp if after reading your critique people go ‘oh yeah, thats obviously true, they’re so right, this couldn’t be interpreted any other way’
on lesswrong, when commenting, a higher value comment is something that adds something new that the post doesnt have.
however, its harder to do this positively in a way that agrees with the post than it is to disagree with the post and say a reason why you disagree.
this biases posts that are just plainly good and dont have much worth critiquing to either just get a bunch of strawman critiques, false critiques, or little to no interaction at all.
Thanks for standing on your morals! did you have any thoughts on striking vs resigning?
fwiw I dont think this is a reason to be pessimistic about governance overall
Avoiding the actual question
Lots of what is being done, all the time, is avoiding difficult questions
Important to notice in ourselves, when we are doing this and in others.
Examples:
Proposing solutions before discussing a hard problem as thoroughly as possible