But no version of Claude has actually beaten the game, so it seems a bit strange to say they’ve crossed half the distance to median human play…
Im not sure I should update on this, since it is from an AI.
Also, it sounds like Gemini has beaten Pokémon with a fixed harness now?? But we don’t know where it was trained on Pokémon?
I agree we probably shouldn’t update much on this, it’s from AI and it’s janky.As for beating the game… well sure, but based on the above graphs it seems like Claude will beat the game within about a year?
Other models have beaten the game months ago, but with more advanced harnesses/scaffolds.
But no version of Claude has actually beaten the game, so it seems a bit strange to say they’ve crossed half the distance to median human play…
Im not sure I should update on this, since it is from an AI.
Also, it sounds like Gemini has beaten Pokémon with a fixed harness now?? But we don’t know where it was trained on Pokémon?
I agree we probably shouldn’t update much on this, it’s from AI and it’s janky.
As for beating the game… well sure, but based on the above graphs it seems like Claude will beat the game within about a year?
Other models have beaten the game months ago, but with more advanced harnesses/scaffolds.