Could one do Efficient Pilot Wave? Mathematically equivalent but you hardly need to compute any of it?
lemonhope
Karma: 1,778
I have been thinking in terms of this fwdperbkwd model of measuring AI agent progress for a few months. This is the first time I actually spelled it out, I just realized. https://www.lesswrong.com/posts/2RwDgMXo6nh42egoC/how-to-game-the-metr-plot?commentId=6fXFqMFxJKdtcZzbs
To clarify my point, I’m saying
A. Standalone/unattended (!) AI agents are clearly much worse than nothing, for both writing code (with a goal chosen in advance) and hacking (a prechosen target), they will clearly only mess up your plans.
If you want to build whatever or hack whoever, and you don’t care what, that’s a different story.
B. If your AI can make productive incremental progress, without messing up old code, writing false notes, citing misleading evidence, breaking infra, alerting the target, falsifying or ruining the tests, fooling the judge, etc, etc, then you can scale that to do basically anything. If you can lay a brick then you can make a cathedral.
If your whole fancy best-of-n-with-rollbacks system is able to lay down bricks, that also counts as OK.
C. I’m betting against the amazing-do-it-all-at-once-from-scratch AI being either a serious risk vector or driver of economic progress.
At least, if you do take over the world on Tuesday morning, it only counts as a victory if you don’t also accidentally give it all back on Tuesday afternoon.
D. I do think these task executions measure something meaningful, and are valuable to conduct, and we should try to extrapolate, but “task horizon length” is definitely not the thing to focus on here. Maybe something like the “zero fatal mistake” rate and the “net-forward-progress geometric mean” or something.
The world just does not look like “task horizons”. The world is made up of steps forward and steps backwards. A scientist or a CEO or an engineer or a horse or a door is measured by how many steps forward it brings you, per step backwards it takes you (which you weren’t able to spot and avoid obviously).
Fwdperbkwd is harder to measure than task horizons, maybe twice as hard, but not four times harder.
The strongest way I could put the punchline here is that I think AI agents will be useless until their code helps more than it hurts, then AI agents will replace coders completely. And you can only measure this by comparing agentless vs agentful approaches to the same task maybe (hey! METR did that too!)
And more ai-live-assistant is-it-worse-than-nothing-or-not experiments would be highly informative in my opinion!
Horizon length makes no sense fundamentally. The 9s of reliability don’t matter for anything but driving a car. Software and hacking and science and engineering are not a series of sequential steps. It is more like adding stuff to your house. If you can make a tiny improvement then you are net-helpful and you can build a cathedral. If you make improvements more often than you cause problems, anything is possible. If you break stuff more often than you fix it, then nothing is possible. It is a binary question. If you make big improvements, then small mistakes don’t matter. If you mess up stuff bigtime and can’t fix it, then big improvements are useless.
I think “AI agent” is here to stay
I have found 0 twitter much better than 0.1 twitter
Hmm maybe you are right about my great-grandparents food availability, I should look into it more.
The second quote was meant to be a joke.
Chemical (hunger) argument paraphrased
I am ordering a motivational poster of this
Eugenics is not metrically fit; it makes your grandma gasp and frown; it does not quickly directly benefit the people who make it happen; it does not have the exciting element of a sudden moment.
I think that the politicians are doing their best to prop up home prices and keep the pensions alive and make everything stay normal etc. Banks try to stop you from withdrawing a lot of money at once. The only problem is that we got used to growth and innovation and freedom and peace and comfort and security as the norm. It is deep in the culture… Scammers and wars usually come from across national borders; what you are asking for is not a simple easy clear thing; I don’t blame you for wishing.
This is really clear and good and I finally understand why I was so bad at basketball.
This immediately came to my mind. And the exact opposite. Any movie recommendations?
I think you are right. The original argument sounds dead to me.
The WWDSC is nearly a consensus. Certainly a plurality.
Please just write the standard library!
Florida man goes insane after accepting conclusion!
Texas woman cleverly navigates around the conclusion!
Raemon found on overpass shouting at cars “just accept the conclusion and deal with it, you can actually do that, that is actually fine, it is just a regular ol conclusion guys, you have dealt with annoying true facts before”
Record-setting interstate pileup as people swerve to avoid the conclusion!
TLDR: a big manipulation trick, maybe the biggest, is to prod people about their own identities and almost-forgotten memories, then get them to pick a stance, then get them to stand for it.
Watch out for: 1: “remember back to when you were a very young child” etc etc; 2: “now if you believe that then act on it” or “remember this next time”
Problem! This is disabling most coordination or acting-on-your-principles etc! Solution!: be reasonable and nonstupid i guess?
I hope that when episodic memory is cracked (ie cheap), it helps with this all-or-nothing problem. That would be really great.