Pekka Puupaa

Karma: 20

Pekka Puupaa 28 Feb 2025 17:20 UTC
4 points
3
in reply to: Cole Wyeth’s comment on: Daniel Kokotajlo’s Shortform
I’m not actually relying on a heuristic, I’m compressing https://www.lesswrong.com/posts/vvgND6aLjuDR6QzDF/my-model-of-what-is-going-on-with-llms
Very interesting, thanks! On a quick skim, I don’t think I agree with the claim that LLMs have never done anything important. I know for a fact that they have written a lot of production code for a lot of companies, for example. And I personally have read AI texts funny or entertaining enough to reflect back on, and AI art beautiful enough to admire even a year later. (All of this is highly subjective, of course. I don’t think you’d find the same examples impressive.) If you don’t think any of that qualifies as important, then I think your definition of important may be overly broad.
But I’ll have to look at this more deeply later.
If you extrapolate log GDP growth or the value of the S&P 500, superintelligence would not be anticipated any time soon. If you extrapolate then number of open mathematical theorems proved by LLMs you get ~a constant at 0. You have to decide which straight line you expect to stay straight—what Aschenbrenner did is not objective, and I don’t know about Kokotajlo but I doubt it was meaningfully independent.
I think this reasoning would also lead one to reject Moore’s law as a valid way to forecast future compute prices. It is in some sense “obvious” what straight lines one should be looking at: smooth lines of technological progress. I claim that you can pick just about any capability with a sufficiently “smooth”, “continuous” definition (i.e. your example of the number of open mathematical theorems solved would have to be amended to allow for partial progress and partial solutions) will tend to converge around 2027-28. Some converge earlier, some later, but that seems to be around the consensus for when we can expect human-level capability for nearly all tasks anybody’s bothered to model.
Interesting, link?
The Mobile Aloha website: https://mobile-aloha.github.io/
The front page has a video of the system autonomously cooking a shrimp and other examples. It is still quite slow and clumsy, but being able to complete tasks like this at all is already light years ahead of where we were just a few years ago.
I’ve worked in robotics research a little bit and I can tell you that setting up a demo for an isolated task is VERY different from selling a product that can do it, let alone one product that can seamlessly transition between many tasks.
Oh, I know. It’s normally 5-20 years from lab to home. My 2027 prediction is for a research robot being able to do anything a human can do in an ordinary environment, not necessarily a mass-producable, inexpensive product for consumers or even most businesses. But obviously the advent of superintelligence, under my model, is going to accelerate those usual 5-20 year timelines quite a bit, so it can’t be much after 2027 that you’ll be able to buy your own android. Assuming “buying things” is still a thing, assuming the world remains recognizable for at least some years, and so on.

Pekka Puupaa 28 Feb 2025 13:48 UTC
6 points
4
in reply to: Cole Wyeth’s comment on: Daniel Kokotajlo’s Shortform
My point is that that heuristic is not good. This obviously doesn’t mean that reversing the heuristic would give you good results (reverse stupidity is not intelligence and so on). What one needs is a different set of heuristics.
If you extrapolate capability graphs in the most straightforward way, you get the result that AGI should arrive around 2027-2028. Scenario analyses (like the ones produced by Kokotajlo and Aschenbrenner) tend to converge on the same result.
An effective cancer cure will likely require superintelligence, so I would be expecting one around 2029 assuming alignment gets solved.
We mostly solved egg frying and laundry folding last year with Aloha and Optimus, which were some of the most long-standing issues in robotics. So human level robots in 2024 would actually have been an okay prediction. Actual human level probably requires human level intelligence, so 2027.

Pekka Puupaa 28 Feb 2025 11:16 UTC
5 points
2
in reply to: Cole Wyeth’s comment on: Daniel Kokotajlo’s Shortform
It’s been over ~40 years of progress since the perceptron, how do you know we’re in the last ~10% today?
What would this heuristic have said about the probability of AlphaFold 2 solving protein folding in 2020? What about all the other tasks that had been untractable for decades that became solvable in the past five years?
To me, 50% over the next 3 years is what sanity looks like.

Pekka Puupaa 17 Feb 2025 3:11 UTC
3 points
3
in reply to: Rafael Harth’s comment on: ≤10-year Timelines Remain Unlikely Despite DeepSeek and o3
Thank you, this has been a very interesting conversation so far.
I originally started writing a much longer reply explaining my position on the interpretation of QM in full, but realized that the explanation would grow so long that it would really need to be its own post. So instead, I’ll just make a few shorter remarks. Sorry if these sound a bit snappy.
As soon as you assume that there exists an external universe, you can forget about your personal experience just try to estimate the length of the program that runs the universe.
And if one assumes an external universe evolving according to classical laws, the Bohmian interpretation has the lowest KC. If you’re going to be baking extra assumptions into your theory, why not go all the way?
Interpretations and Kolmogorov Complexity
An interpretation is still a program. All programs have a KC (although it is usually ill-defined). Ultimately I don’t think it matters whether we call these objects we’re studying theories or interpretations.
Collapse postulate
Has nothing to do with how the universe operates, as I see it. If you’d like, I think we can cast Copenhagen into a more Many Worlds -like framework by considering Many Imaginary Worlds. This is an interpretation, in my opinion functionally equivalent to Copenhagen, where the worlds of MWI are assumed to represent imaginary possibilities rather than real universes. The collapse postulate, then, corresponds to observing that you inhabit a particular imaginary world—observing that that world is real for you at the moment. By contrast, in ordinary MWI, all worlds are real, and observation simply reduces your uncertainty as to which observer (and in which world) you are.
If we accept the functional equivalence between Copenhagen and MIWI, this gives us an upper bound on the KC of Copenhagen. It is at most as complex as MWI. I would argue less.
Chess

I think we need to distinguish between “playing skill” and “positional evaluation skill”. It could be said that DeepBlue is dumber than Kasparov in the sense of being worse at evaluating any given board position than him, while at the same time being a vastly better player than Kasparov simply because it evaluates exponentially more positions.
If you know that a player has made the right move for the wrong reasons, that should still increase your estimate of their playing skill, but not their positional evaluation skill.
Of course, in the case of chess, the two skills will be strongly correlated, and your estimate of the player’s playing skill will still go down as you observe them making blunders in other positions. But this is not always so. In some fields, it is possible to reach a relatively high level of performance using relatively dumb heuristics.
Moving onto the case of logical arguments, playing skill corresponds to “getting the right answers” and positional evaluation skill corresponds to “using the right arguments”.
In many cases it is much easier to find the right answers than to find correct proofs for those answers. For example, most proofs that Euler and Newton gave for their mathematical results are, technically, wrong by today’s standards of rigor. Even worse, even today’s proofs are not completely airtight, since they are not usually machine-verifiable.
And yet we “know” that the results are right. How can that be, if we also know that our arguments aren’t 100% correct? Many reasons, but one is that we can see that our current proofs could be made more rigorous. We can see that they are steelmannable. And in fact, our current proofs were often reached by effectively steelmanning Euler’s and Newton’s proofs.
If we see DeepSeek making arguments that are steelmannable, that should increase our expectation that future models will, in fact, be able to steelman those arguments.

Pekka Puupaa 16 Feb 2025 21:26 UTC
3 points
2
in reply to: Rafael Harth’s comment on: ≤10-year Timelines Remain Unlikely Despite DeepSeek and o3
I don’t believe this is correct.
Which part do you disagree with? Whether or not every interpretation needs a way to connect measurements to conscious experiences, or whether they need extra machinery?
If the former: you need some way to connect the formalism to conscious experiences, since that’s what an interpretation is largely for. It needs to explain how the classical world of your conscious experience is connected to the mathematical formalism. This is true for any interpretation.
If you’re saying that many worlds does not actually need any extra machinery, I guess the most reasonable way to interpret that in my framework is to say that the branching function is a part of the experience function. I suppose this might correspond to what I’ve heard termed the Many Minds interpretation, but I don’t understand that one in enough detail to say.

A bad argument does not improve because there exists a different argument that shares the same conclusion.
Let an argument A be called “steelmannable” if there exists a better argument S with a similar structure and similar assumptions (according to some metric of similarity) that proves the same conclusion as the original argument A. Then S is called a “steelman” of A.
It is clear that not all bad arguments are steelmannable. I think it is reasonable to say that steelmannable bad arguments are less nonsensical than bad arguments that are not steelmannable.
So the question becomes: can my argument be viewed as a steelman of DeepSeek’s argument? I think so. You probably don’t. However, since everybody understands their own arguments quite well, ceteris paribus it should be expected that I am more likely to be correct about the relationship between my argument and DeepSeek’s in this case.
… Or at least, that would be so if I didn’t have an admitted tendency to be too lenient in interpreting AI outputs. Nonetheless, I am not objecting to the claim that DeepSeek’s argument is weak, but to the claim that it is nonsense.
We can both agree that DeepSeek’s argument is not great. But I see glimmers of intelligence in it. And I fully expect that soon we will have models that will be able to argue the same things with more force.

Pekka Puupaa 16 Feb 2025 2:29 UTC
5 points
4
in reply to: Rafael Harth’s comment on: ≤10-year Timelines Remain Unlikely Despite DeepSeek and o3
I am also not a physicist, so perhaps I’ve misunderstood. I’ll outline my reasoning.
An interpretation of quantum mechanics does two things: (1) defines what parts of our theory, if any, are ontically “real” and (2) explains how our conscious observations of measurement results are related to the mathematical formalism of QM.
The Kolmogorov complexity of different interpretations cannot be defined completely objectively, as DeepSeek also notes. But broadly speaking, defining KC “sanely”, it ought to be correlated with a kind of “Occam’s razor for conceptual entities”, or more precisely, “Occam’s razor over defined terms and equations”.
I think Many Worlds is more conceptually complex than Copenhagen. But I view Copenhagen as a catchall term for a category of interpretations that also includes QBism and Rovelli’s RQM. Basically, these are “observer-dependent” interpretations. I myself subscribe to QBism, but I view it as a more rigorous formulation of Copenhagen.
So, why should we think Many Worlds is more conceptually complex? Copenhagen is the closest we can come to a “shut up and calculate” interpretation. Pseudomathematically, we can say
Copenhagen ~= QM + “simple function connecting measurements to conscious experiences”
The reason we can expect Copenhagen-y interpretations to be simpler than other interpretations is because every other interpretation *also* needs a function to connect measurements to conscious experiences, but usually requires some extra machinery in addition to that.
Now I maybe don’t understand MWI correctly. But as I understand it, what QM mathematically gives you is more like a chaotic flux of possibilities, rather than the kind of branching tree of self-consistent worldlines that MWI requires. The way you split up the quantum state into branches constitutes extra structure on top of QM. Thus:
Many Worlds ~= QM + “branching function” + “simple function connecting measurements to conscious experiences”
So it seems that MWI ought to have higher Kolmogorov Complexity than Copenhagen.
I don’t think DeepSeek has argued this point correctly. But I also wouldn’t call its answer nonsense. I would call it a reasonable but superficial answer with a little bit of nonsense mixed in; the kind of answer one might expect a precocious college student to give.
But again, I might be wrong.

Pekka Puupaa 14 Feb 2025 20:01 UTC
1 point
0
on: ≤10-year Timelines Remain Unlikely Despite DeepSeek and o3
Can the author or somebody else explain what is wrong with Deepseek’s answer to the Kolmogorov complexity question? It seems to give more or less the same answer I’d give, and even correctly notes the major caveat in the last sentence of its output.
I suppose its answer is a bit handwavy (“observer role”?), and some of the minor details of its arguments are wrong or poorly phrased, but the conclusion seems correct. Am I misunderstanding something?