I wanted to comment on this, not because I think it’ll answer your questions, but because it lies at the center of my interests, and you nerd-sniped me. Anyway, sorry for just now getting around to it.
I’m longtime (pretty average) go player, and interested in things like ‘intelligence’ and ‘consciousness’ (in humans really, but lately computers are also interesting)
I watched the Lee Sedol live streams, and had a lot of fun, learning about human nature.
I actually gave a very short seminar talk about them and related things once, because to me they are a very distilled example of something that I also noticed in your post.
Anyway, here goes.
I frequently use “Move 37” as a shorthand for “AI that comes up with creative, highly effective ideas that no human would ever consider.”
What does ‘creative’ mean? Which humans? All of them, even both the pros, and the village idiots?
Considering how influential Move 37 is on my views about AI, it seems like I’d better try to understand what was so special about it.
Why is this move so important to you? Why do people buy overpriced sneakers? Because they heard other people talking about them?
Regarding the understanding part:
I think it is important to distinguish between different time frames. e.g. explanations that come before the event, or, minutes, days, or years after etc.
For example, before AlphaGo, it was common knowledge, that computers will not beat Go for a long time, because they have no ‘intuition’. (Similar to how they would never create art, because they had no creativity, etc.)
They first thing I learned here, is that people (apart from mathematicians, and a small number of physicists) never define their terms properly. (Neither did you, above)
Now, afterwards, since Lee Sedol lost, Alphago necessarily has to have intuition or creativity, or whatever was missing before. But this does not mean anything. I guess it’s called Wittgenstein’s ruler. You are not measuring the capabilities of AlphaGo relative to your (well defined) concepts, but your (ill-defined) concepts get retrofitted and filled with meaning by what AlphaGo did.
For an example at what this looks like in action, please see the game itself:
From that timestamp, for about a minute or two.
There are two commentators, which for live TV Go games is the standard format. In this case, on the right, Redmond is the ‘smartest person in the room’, and on the left, the ‘jester’, who acts the fool and asks questions, which Redmond answers for the benefit of the audience. Without them, a normal person (and many players) would not be able to understand what the fuck is happening, and who is winning (even for any other televised game). Anyway, the smart one normally is of the same level as the players, and really the only person who can follow all the details. They always understand what is going on. Not here though.
Watch Redmond. He places the stone and then just moves it somewhere else, because obviously it’s wrong there and has to have been a ‘misclick’ by the human inputting the moves. It resembles a commonly known shape, a ‘shoulder hit’, but is farther from the edge than normally.
Anyway, they and other commentators go from ‘misclick’ to ‘I don’t understand’ to ‘this is known to be bad’, to ’huh whadda ya know’
What I’m trying to say is that the smartest people in the room just make mouth noises since their heuristics no longer work. Normally, whatever the commentator says is law, since there is no one stronger present to correct them.
Again, here you don’t learn anything about good Go moves (Since no one present understands at that level). You learn about human experts learning in real time.
I’m mainly interested in trying to understand the intuition behind Move 37 in the way AlphaGo might have “understood” it
Your whole post is full of sentences like this. What does ‘intuition’ mean? I think you are going at it wrong. AlphaGo just ‘knows’ more about Go (See, now I did it). There are no shortcuts. Out with the old (knowledge). In with the new.
When we have superintelligent AI solving real-world problems using strategies that no human would ever think of, those strategies should ideally be explainable, if not in practice, at least in principle—perhaps even to the point that a human could understand and replicate the strategies given enough time to study the explanation.
Egyptians did not know how to draw using the correct perspective, and now we do. They didn’t know how to do complex quantum chemical calculations either, and now we can. Still, one is more difficult (in invested energy) than the other, and I don’t see how to distinguish one from the other a priori.
Maybe it’s hubris to hope that a complete novice like me could understand anything about it, but I’d be surprised if it weren’t possible to get some intuition for why this move was important.
I don’t think the move was important, to be honest. The event, the demonstration, the technological leap certainly was. As in someone dying today versus tomorrow. The date is not the important part. The dying versus not/never dying is.
Again, your focus (to me) seems to be on the less important things.
For example, if you look at the comments on LLM technology, have in mind the different timescales. Comments on fresh new things are different than comments that come years later. And you can learn a lot from how people comment on these things. Just like when Redmond automatically moved the stone, because he ‘knew’ where it should really be.
Does understanding Move 37 require the use of extensive brute force search?
brute? no, extensive, yes. Humans (playing many many games) found many recurring useful patterns, but it seems not all.
When you take into account the fact that AlphaGo had extensive search at its disposal, does that make the creativity of Move 37 significantly less impressive?
It makes it more probable, which is why it found it. the word ‘creativity’ has to be defined before this question can be answered.
I noticed that Lee Sedol’s Wikipedia page mentions a notable game in which he uses a “broken ladder,” which is “associated with beginner play”—maybe it’s not so uncommon for a professional Go player to do something unconventional every so often.
That game is a good example. It’s basically a tradeoff. A professional would not typically end up in that situation, because the search tree involves a few ‘obviously bad’ moves before one profits. Maybe as in ‘two wrongs make it right’, and the pro just truncates the search after the first wrong if you will.
Still, it’s easily understandable after the fact, or with very good foresight. it’s just that it’s maybe outside your normal heuristics.
Given an expert explanation of Move 37, what level of Go expertise would be required to fully understand it, and how long would it take?
There are only ‘just-so’, handwaving after-the-fact explanations (of the type seen in the video). There is no absolute truth (yet). The next iteration of Go playing robot will invalidate them, just as AlphaGo did (until the game is eventually completely solved)
To understand you just have to play (a lot in this case). Otherwise you’ll just be like someone that wants to learn Parcour or Kung Fu by watching Youtube videos, and never once moving a muscle. Metis vs Episteme I guess.
To what extent have human players been able to learn novel strategies from AI in Go or chess?
There are many new moves that entered common knowledge, particularly in the opening. For example, this one comes up a lot in games at my level and has changed due to alphaGo
https://www.josekipedia.com/#path:pdqcqdpcocobncnbmc
As a result, other variations basically disappeared completely. but not because they would loose you the game, but because we have been told they are now bad, and we (average amateurs) simply lack the skill to understand every nuance.
Broadly speaking, how is an advanced AI’s “playstyle” different from advanced human players?
I guess people called it alien or strange, or other words, but really all these words mean ‘I don’t (yet) understand why this move is good’. Once they do, AI will just play ‘normal’ again.
Once human Metis catches up (at least in things like openings, where few moves are involved), thing cool down again.
In the end it’s just people gossiping. You learn not by memorizing what they say, but by understanding why they say it, or why others stay silent.
Are you familiar with Erik Hoel? I get the feeling you are not, yet should be. He is writing at https://www.theintrinsicperspective.com/
He wrote several pieces arguing against your core pillars (The anthropic papers/grandiose sounding press releases on emotions, introspection, workspace). His basic argument is that there is something there, but there is absolutely no need to use anthropomorphizing language, since the data is not that conclusive/ no proper control experiments are conducted.
Another one of his theses is, that there will not be any (falsifiable and non-trivial) theories of consciousness, unless they require continual learning https://arxiv.org/abs/2512.12802
Is an encyclopedia conscious? What if it is really large, and organized in a high-dimensional manner?
If yes, then you are just playing with words (a trivial, useless definition). If no, why should a static LLM be conscious?
More interestingly, how complex of an update/learning process is needed, before LLMs are indeed conscious? (I don’t know if he addresses that explicitly, though I don’t think some .md files count)
Relatedly, I do not think LLMs are strange loopy at all. In my opinion, they would be if they could update themselves, which they don’t do yet.
LLMs are as strange loopy, as a dead human in a brain-scanner. Static. Not the same thing as a live human.
Re: “models believe they are conscious”:
What is the important part? That they say it? Would you believe any entity that claims such? Do you have to check? Do you have to be able to check, at least in principle? If checking is required, what takes precedence? The self report, or the result of the check? I see no way out of this that does not result in circular reasoning or other logical problems.
Self-reports simply are not useful to resolve this question.
Another person with a strong opinion on this topic is Joscha Bach https://cimc.ai/
I guess I’ll butcher his position even worse than Erik’s, but I would summarize it as:
Consciousness is a pattern or mechanism that allows to learn, and therefore to act. It may be the simplest way to train a self-organizing system.
Looking at GPU and RAM and power prices lately, I don’t think LLMs use the simplest of all possible methods right now.
My personal opinion is that all this LLM development goes in the direction of “more” consciousness, but we are not there yet. At least not at a human-comparable version of it. It may well be something like an instantaneous snapshot of a somewhat alien consciousness.