Ted Sanders

Karma: 202

Ted Sanders 6 Jun 2023 23:46 UTC
44 points
−1
in reply to: Daniel Kokotajlo’s comment on: Transformative AGI by 2043 is <1% likely
Oh, to clarify, we’re not predicting AGI will be achieved by brain simulation. We’re using the human brain as a starting point for guessing how much compute AGI will need, and then applying a giant confidence interval (to account for cases where AGI is way more efficient, as well as way less efficient). It’s the most uncertain part of our analysis and we’re open to updating.

For posterity, by 2030, I predict we will not have:
- AI drivers that work in any country
- AI swim instructors
- AI that can do all of my current job at OpenAI in 2023
- AI that can get into a 2017 Toyota Prius and drive it
- AI that cleans my home (e.g., laundry, dishwashing, vacuuming, and/or wiping)
- AI retail workers
- AI managers
- AI CEOs running their own companies
- Self-replicating AIs running around the internet acquiring resources
Here are some of my predictions from the past:
- Predictions about the year 2050, written 7ish years ago: https://www.tedsanders.com/predictions-about-the-year-2050/
- Predictions on self-driving from 5 years ago: https://www.tedsanders.com/on-self-driving-cars/

Transformative AGI by 2043 is <1% likely

Ted Sanders6 Jun 2023 17:36 UTC

34 points

115 comments5 min readLW link

(arxiv.org)

Ted Sanders 7 Jun 2023 7:44 UTC
27 points
16
in reply to: Daniel Kokotajlo’s comment on: Transformative AGI by 2043 is <1% likely
Great points.
I think you’ve identified a good crux between us: I think GPT-4 is far from automating remote workers and you think it’s close. If GPT-5/6 automate most remote work, that will be point in favor of your view, and if takes until GPT-8/9/10+, that will be a point in favor of mine. And if GPT gradually provides increasingly powerful tools that wildly transform jobs before they are eventually automated away by GPT-7, then we can call it a tie. :)
I also agree that the magic of GPT should update one into believing in shorter AGI timelines with lower compute requirements. And you’re right, this framework anchored on the human brain can’t cleanly adjust from such updates. We didn’t want to overcomplicate our model, but perhaps we oversimplified here. (One defense is that the hugeness of our error bars mean that relatively large updates are needed to make a substantial difference in the CDF.)
Lastly, I think when we see GPT unexpectedly pass the Bar, LSAT, SAT, etc. but continue to fail at basic reasoning, it should update us into thinking AGI is sooner (vs a no pass scenario), but also update us into realizing these metrics might be further from AGI than we originally assumed based on human analogues.

Ted Sanders 6 Jun 2023 23:52 UTC
17 points
7
in reply to: Gerald Monroe’s comment on: Transformative AGI by 2043 is <1% likely
We invent a way for AGIs to learn faster than humans : Why is this even in the table? This would be 1.0 because it’s a known fact, AGI learns faster than humans. Again, from the llama training run, the model went from knowing nothing to domain human level in 1 month. That’s faster. (requiring far more data than humans isn’t an issue)
100% feels overconfident. Some algorithms learning some things faster than humans is not proof that AGI will learn all things faster than humans. Just look at self-driving. It’s taking AI far longer than human teenagers to learn.
AGI inference costs drop below $25/hr (per human equivalent): Well, A100s are 0.87 per hour. A transformative AGI might use 32 A100s. $27.84 an hour. Looks like we’re at 1.0 on this one also.
100% feels overconfident. We don’t know if transformative will need 32 A100s, or more. Our essay explains why we think it’s more. Even if you disagree with us, I struggle to see how you can be 100% sure.

Ted Sanders 13 Jun 2023 7:56 UTC
12 points
0
in reply to: jimrandomh’s comment on: Transformative AGI by 2043 is <1% likely
Interested in betting thousands of dollars on this prediction? I’m game.

Ted Sanders 20 Aug 2023 21:14 UTC
9 points
6
in reply to: Matt Goldenberg’s comment on: AI Forecasting: Two Years In
Strongly disagree. Employees of OpenAI and their alpha tester partners have obligations not to reveal secret information, whether by prediction market or other mechanism. Insider trading is not a sin against the market; it’s a sin against the entity that entrusted you with private information. If someone tells me information under an NDA, I am obligated not to trade on that information.

Ted Sanders 7 Jun 2023 8:00 UTC
9 points
3
in reply to: RobertM’s comment on: Transformative AGI by 2043 is <1% likely
Interesting. When I participated in the AI Adversarial Collaboration Project, a study funded by Open Philanthropy and executed by the Forecasting Research Institute, I got the sense that most folks concerned about AI x-risk mostly believed that AGIs would kill us on their own accord (rather than by accident or as a result of human direction), that AGIs would have self-preservation goals, and therefore AGIs would likely only kill us after solving robotic supply chains (or enslaving/manipulating humans, as I argued as an alternative).
Sounds like your perception is that LessWrong folks don’t think robotic supply chain automation will be a likely prerequisite to AI x-risk?

Ted Sanders 7 Jun 2023 0:03 UTC
7 points
0
in reply to: Steven Byrnes’s comment on: Transformative AGI by 2043 is <1% likely
Excellent points. Agree that the compute needed to simulate a thing is not equal to the compute performed by that thing. It’s very possible this means we’re overestimating the compute performed by the human brain a bit. Possible this is counterbalanced by early AGIs being inefficient, or having architectural constraints that the human brain lacks, but who knows. Very possible our 16% is too low, and should be higher. Tripling it to ~50% would yield a likelihood of transformative AGI of ~1.2%.

Ted Sanders 22 Jun 2023 22:06 UTC
6 points
2
in reply to: RatsWrongAboutUAP’s comment on: UFO Betting: Put Up or Shut Up
Fair. I accept. 200:1 of my $100k against your $500. How are you setting these up?

I’m happy to pay $100k if my understanding of the universe (no aliens, no supernatural, etc.) is shaken. Also happy to pay up after 5 years if evidence turns up later about activities before or in this 5-year period.

(Also, regarding history, I have a second Less Wrong account with 11 years of history: https://www.lesswrong.com/users/tedsanders)

Ted Sanders 9 Jun 2023 21:39 UTC
6 points
0
in reply to: jimrandomh’s comment on: Transformative AGI by 2043 is <1% likely
What conditional probabilities would you assign, if you think ours are too low?

Ted Sanders 7 Jun 2023 8:04 UTC
6 points
0
in reply to: Steven Byrnes’s comment on: Transformative AGI by 2043 is <1% likely
If humans can teleoperate robots, why don’t we have low-wage workers operating robots in high-wage countries? Feels like a win-win if the technology works, but I’ve seen zero evidence of it being close. Maybe Ugo is a point in favor?

Ted Sanders 7 Jun 2023 0:08 UTC
5 points
0
in reply to: Dagon’s comment on: Transformative AGI by 2043 is <1% likely
How has this forecast changed in the last 5 years? Has widespread and rapid advance of non-transformative somewhat-general-purpose LLMs change any of your component predictions?
We didn’t have this framework 5 years ago, but the tremendous success of LLMs can only be a big positive update, I think. That said, some negative updates for me from the past 15 years have been how slowly Siri improved, how slowly Wolfram Alpha improved, and how slowly Alexa improved. I genuinely expected faster progress from their data flywheels after their launches, but somehow it didn’t seem to happen. Self-driving seems to be middle of the road compared to how I thought it would go 5 years ago.
I don’t actually disagree, but MUCH of the cause of this is an excessively high bar (as you point out, but it still makes the title misleading).
Agreed. I think the “<1%” headline feels like an aggressive claim, but the definition from the contest we use is a very high bar. For lower bars, we’d forecast much higher probabilities. We expect great things from AI and AGI, and we are not reflexively bearish on progress.

Ted Sanders 23 Jun 2023 22:37 UTC
4 points
2
in reply to: RatsWrongAboutUAP’s comment on: UFO Betting: Put Up or Shut Up
$500 payment received.
I am committed to paying $100k if aliens/supernatural/non-prosaic explanations are, in the next 5 years, considered, in aggregate, to be 50%+ likely in explaining at least one UFO.

Ted Sanders 19 Jun 2023 5:25 UTC
4 points
0
on: UFO Betting: Put Up or Shut Up
I’ll bet. Up to $100k of mine against $2k of yours. 50:1. (I honestly think the odds are more like 1000+:1, and would in principle be willing to go higher, but generally think people shouldn’t bet more than they’d be willing to lose, as bets above that amount could drive bad behavior. I would be happy to lose $100k on discovering aliens/time travel/new laws of physics/supernatural/etc.)
Happy to write a contract of sorts. I’m a findable figure and I’ve made public bets before (e.g., $4k wagered on AGI-fueled growth by 2043).

Ted Sanders 14 Jun 2023 0:12 UTC
4 points
1
in reply to: Ege Erdil’s comment on: Transformative AGI by 2043 is <1% likely
Thanks, that’s clarifying. (And yes, I’m well aware that x → B*x is almost never injective, which is why I said it wouldn’t cause 8 bits of erasure rather than the stronger, incorrect claim of 0 bits of erasure.)
To store 1 bit of information you need a potential energy barrier that’s at least as high as k_B T log(2), so you need to switch ~ 8 such barriers, which means in any kind of realistic device you’ll lose ~ 8 k_B T log(2) of electrical potential energy to heat, either through resistance or through radiation. It doesn’t have to be like this, and some idealized device could do better, but GPUs are not idealized devices and neither are brains.
Two more points of confusion:
- Why does switching barriers imply that electrical potential energy is probably being converted to heat? I don’t see how that follows at all.
- To what extent do information storage requirements weigh on FLOPS requirements? It’s not obvious to me that requirements on energy barriers for long-term storage in thermodynamic equilibrium necessarily bear on transient representations of information in the midst of computations, either because the system is out of thermodynamic equilibrium or because storage times are very short

Ted Sanders 7 Jun 2023 22:48 UTC
4 points
3
in reply to: Charlie Steiner’s comment on: Transformative AGI by 2043 is <1% likely
I’m curious and I wonder if I’m missing something that’s obvious to others: What are the algorithms we already have for AGI? What makes you confident they will work before seeing any demonstration of AGI?

Ted Sanders 7 Jun 2023 7:52 UTC
4 points
0
in reply to: AnthonyC’s comment on: Transformative AGI by 2043 is <1% likely
According to our rough and imperfect model, dropping inference needs by 2 OOMs increases our likelihood of hitting the $25/hr target by 20%abs, from 16% to 36%.
It doesn’t necessarily make a huge difference to chip and power scaling, as in our model those are dominated by our training estimates, not our inference need estimates. (Though of course those figures will be connected in reality.)
With no adjustment to chip and power scaling, this yields a 0.9% likelihood of TAGI.
With a +15%abs bump to chip and power scaling, this yields a 1.2% likelihood of TAGI.

Ted Sanders 20 Aug 2023 5:09 UTC
3 points
0
on: AI Forecasting: Two Years In
Thanks for the write up. I was a participant in both Hypermind and XPT, but I recused myself from the MMLU question (among others) because I knew the GPT-4 result many months before the public. I’m not too surprised Hypermind was the least accurate—I think the traders there are less informed, plus the interface for shaping the distribution is a bit lacking (my recollection is that last year’s version capped the width of distributions which massively constrained some predictions). I recall they also plotted the current values, a generally nice feature which has the side effect of anchoring ignorant forecasters downward, I’d bet.
Question: Are the Hypermind results for 2023 just from forecasts in 2022, or do they include forecasts from the prior year as well? I’m curious if part of the poor accuracy is from stale forecasts that were never updated.

Ted Sanders 14 Jun 2023 6:59 UTC
3 points
0
in reply to: Ege Erdil’s comment on: Transformative AGI by 2043 is <1% likely
No worries. I’ve made far worse. I only wish that H100s could operate at a gentle 70 W! :)

Ted Sanders 14 Jun 2023 6:55 UTC
3 points
0
in reply to: Ege Erdil’s comment on: Transformative AGI by 2043 is <1% likely
I think what I don’t understand is why you’re defaulting to the assumption that the brain has a way to store and update information that’s much more efficient than what we’re able to do. That doesn’t sound like a state of ignorance to me; it seems like you wouldn’t hold this belief if you didn’t think there was a good reason to do so.
It’s my assumption because our brains are AGI for ~20 W.
In contrast, many kW of GPUs are not AGI.
Therefore, it seems like brains have a way of storing and updating information that’s much more efficient than what we’re able to do.
Of course, maybe I’m wrong and it’s due to a lack of training or lack of data or lack of algorithms, rather than lack of hardware.
DNA storage is way more information dense than hard drives, for example.

Ted Sanders

Trans­for­ma­tive AGI by 2043 is <1% likely

Transformative AGI by 2043 is <1% likely