Just for fun, I tried solving the sliding block puzzle pretending I was an LLM (no drawing pictures, no copy and paste, only writing text).
It was hard in an interesting way: I repeatedly had to say “oops, I made a mistake.”
The mistakes I made were familiar mistakes that anyone could make, but the were also the kind of mistake for which I would never write “oops I made a mistake” unless I’m pretending to be an LLM. They were the kind of mistake that would normally cause me to erase and delete the mistake and simply write the correction, since they were far too uninteresting to save.
My human chain-of-thought
I have the following pieces in this order:
Sun Leaf (Empty)
Bell Snake Star
(Empty) Flower (Empty)
each can be moved into an orthogonally adjacent empty square. Target:
Snake Star Flower
(Empty) Sun (Empty)
Leaf (Empty) Bell
, find the smallest number of moves for getting them from the initial configuration to the target one.
As an actual human, let me try to figure this one out.
First, Sun moves from the top left corner to the middle.
Leaf moves from the center top to the bottom left.
Bell moves from the center left to the bottom right.
Snake moves from the center to the top left.
Star moves from the center right to the center top.
Flower moves from the bottom to the top right.
As an actual human, I realize it is hard to keep track of all these targets because I know when I feel a bit confused. So I will no try to keep track of all these motions at once, but change strategy to moving the pieces based on guesses until I find a solution, and hopefully I can argue the solution is the shortest.
So let me move a piece very far from its target in the general direction of its target, this move is unlikely to need to be reversed.
Let me try moving Bell down. This creates a spot at the center left, and Bell is now at the bottom left corner.
I could move Sun down or Snake left now. Both will get closer to their target. Moving Snake Left opens a spot in the middle, which can be used to ferry pieces to other connected empty spots. So I will move Snake left. Now Snake is at the center left, and there is a spot in the center.
I can use the empty spot in the middle to move pieces. I can move Star to its destination, or move Leaf or Flower close to their destinations, but I’m not sure where to put them so I will move Star first. I move Star from the center right to the middle, then from the middle… up? Oops I made a mistake, there is no empty space at the top because Leaf is there.
Maybe I won’t move Star at all. Maybe, backtracking to before I tried to move Star, I will move Flower right from the center bottom to the bottom right. This opens a spot to move Leaf to its position.
Next I will move Leaf down from the center top to the middle, and then move Leaf down again from the middle to the center bottom, and then move Leaf again from the center bottom to the bottom left.
To recap, I first move Bell down, then move Snake left, then move Flower right, and oh no I spotted another mistake. I cannot move Leaf to its target position because Bell is blocking it.
Let me try to draw the situation before I moved Flower, so I can better keep track of what is happening.
Sun Leaf (Empty)
Snake (Empty) Star
Bell Flower (Empty)
At this point, it looks like I should fill some of the empty corners to make it easier to move other things. I will move Star up, since that moves it closer to its destination.
At this point, Leaf is blocking Star from moving to where it should go, Flower is blocking Leaf from moving to where it should go, and a combination of Leaf and Star is blocking Flower from moving to where it should go.
Oh wait, I made a mistake. I forgot that I moved Star up. As a human, I am aware of what visual model I used to describe things, and noticed it did not include moving Star up.
So now let me redescribe it. Leaf is still blocking Star, Flower and Bell are blocking Leaf, I somehow forgot to mention Bell, and Star is blocking Flower.
I spot a solution. Move Leaf down, then move Star left into its place, then move Flower right, move Flower up, and move Flower up again into its place.
To recap, after moving Bell down and Snake left, I move Star up, I move Leaf down, I move Star left, I move Flower right, I move Flower up, and I move Flower up again.
Let me draw the new situation so I can better keep track of what is happening. As a human I am able to realize when I lose track of the state.
Sun Flower (Empty)
Snake Leaf
Oops, I drew it wrong, let me try drawing it again. I forgot that Star should be in the center top while Flower should be in the top right.
Sun Star Flower
Snake Leaf (Empty)
Bell (Empty) (Empty)
As a human, I have to scroll up in order to see the target state, so I will copy the target state again to avoid having to scroll up in order to see the two pictures at once. I won’t use copy and paste because that may be considered cheating, it would let me create an image of each move which would fail to demonstrate the difficulties of solving this problem using only text and not much visual aids.
The target was:
Snake Star Flower
(Empty) Sun (Empty)
Leaf (Empty) Bell
Alright, I realize there is a problem here. Snake only needs to move down by one, but Sun needs to move to where Snake is. There is no way for Sun to get there unless some pieces move away from their target. It is likely that this is necessary, but if it does occur it will be much harder to prove I moved the pieces to their destination in as few moves as possible.
Therefore, maybe I should backtrack to earlier. After moving only Bell down and Snake left, it looked like this:
Sun Leaf (Empty)
Snake (Empty) Star
Bell Flower (Empty)
So I want to move. Wait, what was I talking about? I somehow mixed up my current state with the target. The above paragraphs may be nonsense then.
Actually, it still makes sense even if I mixed up the current state and target. Basically, the Sun is trapped in the corner.
So let me return to the earlier state. I think I might still do the same strategy of moving Flower in its L shape path and moving Star up and left, but I will do this slowly, and make sure Sun gets out before Star moves to its destination.
I will move Flower right, and move. Oh yes, I also don’t want to trap Bell there in its corner if I moved Leaf down.
So after moving Flower right, I move Star up, and move Flower up. Next I move Bell right and move Bell right again so it is in a corner and out of the way.
To recap, after moving Bell down and Snake left, I move Flower right, Star up, Flower up, Bell right, and Bell right.
I will draw it again, first I visually verify my last drawing was the state after moving Bell down and Snake left.
Now I draw it.
Sun Leaf Star
Snake (Empty) Flower
(Empty) (Empty) Bell
To reach the target, Star and Flower still need to “rotate counterclockwise” by one unit. I can understand this visually because I am a human. Leaf is in their way, but can move to its own destination without issue due to the path.
Sun can move to its destination as soon as Leaf is out of the way, but before Star and Flower rotate to their positions. As soon as Sun is there, Snake can easily get to its spot. It can be seen that once the Sun and Leaf “train” moves, the Star and Flower train can move, as can Snake, the order here is not that important.
Therefore, I will move Leaf down, move Leaf down again, move Leaf left. I will then move Sun right, and move Sun down. Afterwards I will move Star left, and move Flower up. Finally I will move Snake up.
The result should look like the target, but let me draw it to verify.
Snake Star Flower
(Empty) Sun (Empty)
Leaf (Empty) Bell
Yes, this is the target. So in summary, the moves I made were:
Bell down, Snake left, Flower right, Star up, Flower up, Bell right, Bell right, Leaf down, Leaf down, Leaf left, Sun right, Sun down, Star left, Flower up, Snake up.
As a sanity check, I will redraw the original image, and verify that each block ends up at its final decision. I might also try to visualize possible collisions, but this is akin to restating everything and hard to do mentally.
Sun Leaf (Empty)
Bell Snake Star
(Empty) Flower (Empty)
Yes, sun moves down and right to its target. Leaf moves down twice and left into its target. Bell moves down and right twice into its target. Snake moves left and up into its target. Star moves up and left into its target. Flower moves right and up twice into its target.
If there are no collisions, this solution is the shortest since every piece only moves in the direction towards its target.
I visualized the process and there seems to be no collisions, but it was very fuzzy. I am 80% confident this is a right answer.
Bell down, Snake left, Flower right, Star up, Flower up, Bell right, Bell right, Leaf down, Leaf down, Leaf left, Sun right, Sun down, Star left, Flower up, Snake up.
The smallest number of moves is 15.
Edit: I tested o3 (not mini) with the same prompt and it solved it by writing a Breadth First Search from scratch (in Python).
If you’re reading the text in a two-dimensional visual display, you are giving yourself an advantage over the LLM. You should actually be reading it in a one-dimensional format with new-line symbols.
(disclosure, I only skimmed your COT for like a few seconds)
Yeah, I definitely didn’t remove all of my advantages. Another unfair thing I did was I did correct my typos, including accidentally writing the wrong label, when I decided that “I thought the right label, so I’m allowed to correct what I wrote into what I was thinking about.”
Just for fun, I tried solving the sliding block puzzle pretending I was an LLM (no drawing pictures, no copy and paste, only writing text).
It was hard in an interesting way: I repeatedly had to say “oops, I made a mistake.”
The mistakes I made were familiar mistakes that anyone could make, but the were also the kind of mistake for which I would never write “oops I made a mistake” unless I’m pretending to be an LLM. They were the kind of mistake that would normally cause me to erase and delete the mistake and simply write the correction, since they were far too uninteresting to save.
My human chain-of-thought
As an actual human, let me try to figure this one out.
First, Sun moves from the top left corner to the middle.
Leaf moves from the center top to the bottom left.
Bell moves from the center left to the bottom right.
Snake moves from the center to the top left.
Star moves from the center right to the center top.
Flower moves from the bottom to the top right.
As an actual human, I realize it is hard to keep track of all these targets because I know when I feel a bit confused. So I will no try to keep track of all these motions at once, but change strategy to moving the pieces based on guesses until I find a solution, and hopefully I can argue the solution is the shortest.
So let me move a piece very far from its target in the general direction of its target, this move is unlikely to need to be reversed.
Let me try moving Bell down. This creates a spot at the center left, and Bell is now at the bottom left corner.
I could move Sun down or Snake left now. Both will get closer to their target. Moving Snake Left opens a spot in the middle, which can be used to ferry pieces to other connected empty spots. So I will move Snake left. Now Snake is at the center left, and there is a spot in the center.
I can use the empty spot in the middle to move pieces. I can move Star to its destination, or move Leaf or Flower close to their destinations, but I’m not sure where to put them so I will move Star first. I move Star from the center right to the middle, then from the middle… up? Oops I made a mistake, there is no empty space at the top because Leaf is there.
Maybe I won’t move Star at all. Maybe, backtracking to before I tried to move Star, I will move Flower right from the center bottom to the bottom right. This opens a spot to move Leaf to its position.
Next I will move Leaf down from the center top to the middle, and then move Leaf down again from the middle to the center bottom, and then move Leaf again from the center bottom to the bottom left.
To recap, I first move Bell down, then move Snake left, then move Flower right, and oh no I spotted another mistake. I cannot move Leaf to its target position because Bell is blocking it.
Let me try to draw the situation before I moved Flower, so I can better keep track of what is happening.
Sun Leaf (Empty)
Snake (Empty) Star
Bell Flower (Empty)
At this point, it looks like I should fill some of the empty corners to make it easier to move other things. I will move Star up, since that moves it closer to its destination.
At this point, Leaf is blocking Star from moving to where it should go, Flower is blocking Leaf from moving to where it should go, and a combination of Leaf and Star is blocking Flower from moving to where it should go.
Oh wait, I made a mistake. I forgot that I moved Star up. As a human, I am aware of what visual model I used to describe things, and noticed it did not include moving Star up.
So now let me redescribe it. Leaf is still blocking Star, Flower and Bell are blocking Leaf, I somehow forgot to mention Bell, and Star is blocking Flower.
I spot a solution. Move Leaf down, then move Star left into its place, then move Flower right, move Flower up, and move Flower up again into its place.
To recap, after moving Bell down and Snake left, I move Star up, I move Leaf down, I move Star left, I move Flower right, I move Flower up, and I move Flower up again.
Let me draw the new situation so I can better keep track of what is happening. As a human I am able to realize when I lose track of the state.
Sun Flower (Empty)
Snake Leaf
Oops, I drew it wrong, let me try drawing it again. I forgot that Star should be in the center top while Flower should be in the top right.
Sun Star Flower
Snake Leaf (Empty)
Bell (Empty) (Empty)
As a human, I have to scroll up in order to see the target state, so I will copy the target state again to avoid having to scroll up in order to see the two pictures at once. I won’t use copy and paste because that may be considered cheating, it would let me create an image of each move which would fail to demonstrate the difficulties of solving this problem using only text and not much visual aids.
The target was:
Snake Star Flower
(Empty) Sun (Empty)
Leaf (Empty) Bell
Alright, I realize there is a problem here. Snake only needs to move down by one, but Sun needs to move to where Snake is. There is no way for Sun to get there unless some pieces move away from their target. It is likely that this is necessary, but if it does occur it will be much harder to prove I moved the pieces to their destination in as few moves as possible.
Therefore, maybe I should backtrack to earlier. After moving only Bell down and Snake left, it looked like this:
Sun Leaf (Empty)
Snake (Empty) Star
Bell Flower (Empty)
So I want to move. Wait, what was I talking about? I somehow mixed up my current state with the target. The above paragraphs may be nonsense then.
Actually, it still makes sense even if I mixed up the current state and target. Basically, the Sun is trapped in the corner.
So let me return to the earlier state. I think I might still do the same strategy of moving Flower in its L shape path and moving Star up and left, but I will do this slowly, and make sure Sun gets out before Star moves to its destination.
I will move Flower right, and move. Oh yes, I also don’t want to trap Bell there in its corner if I moved Leaf down.
So after moving Flower right, I move Star up, and move Flower up. Next I move Bell right and move Bell right again so it is in a corner and out of the way.
To recap, after moving Bell down and Snake left, I move Flower right, Star up, Flower up, Bell right, and Bell right.
I will draw it again, first I visually verify my last drawing was the state after moving Bell down and Snake left.
Now I draw it.
Sun Leaf Star
Snake (Empty) Flower
(Empty) (Empty) Bell
To reach the target, Star and Flower still need to “rotate counterclockwise” by one unit. I can understand this visually because I am a human. Leaf is in their way, but can move to its own destination without issue due to the path.
Sun can move to its destination as soon as Leaf is out of the way, but before Star and Flower rotate to their positions. As soon as Sun is there, Snake can easily get to its spot. It can be seen that once the Sun and Leaf “train” moves, the Star and Flower train can move, as can Snake, the order here is not that important.
Therefore, I will move Leaf down, move Leaf down again, move Leaf left. I will then move Sun right, and move Sun down. Afterwards I will move Star left, and move Flower up. Finally I will move Snake up.
The result should look like the target, but let me draw it to verify.
Snake Star Flower
(Empty) Sun (Empty)
Leaf (Empty) Bell
Yes, this is the target. So in summary, the moves I made were:
Bell down, Snake left, Flower right, Star up, Flower up, Bell right, Bell right, Leaf down, Leaf down, Leaf left, Sun right, Sun down, Star left, Flower up, Snake up.
As a sanity check, I will redraw the original image, and verify that each block ends up at its final decision. I might also try to visualize possible collisions, but this is akin to restating everything and hard to do mentally.
Sun Leaf (Empty)
Bell Snake Star
(Empty) Flower (Empty)
Yes, sun moves down and right to its target. Leaf moves down twice and left into its target. Bell moves down and right twice into its target. Snake moves left and up into its target. Star moves up and left into its target. Flower moves right and up twice into its target.
If there are no collisions, this solution is the shortest since every piece only moves in the direction towards its target.
I visualized the process and there seems to be no collisions, but it was very fuzzy. I am 80% confident this is a right answer.
Bell down, Snake left, Flower right, Star up, Flower up, Bell right, Bell right, Leaf down, Leaf down, Leaf left, Sun right, Sun down, Star left, Flower up, Snake up.
The smallest number of moves is 15.
Edit: I tested o3 (not mini) with the same prompt and it solved it by writing a Breadth First Search from scratch (in Python).
If you’re reading the text in a two-dimensional visual display, you are giving yourself an advantage over the LLM. You should actually be reading it in a one-dimensional format with new-line symbols.
(disclosure, I only skimmed your COT for like a few seconds)
Yeah, I definitely didn’t remove all of my advantages. Another unfair thing I did was I did correct my typos, including accidentally writing the wrong label, when I decided that “I thought the right label, so I’m allowed to correct what I wrote into what I was thinking about.”