Humans would draw a map of the world from memory, overlay the grid and look up the reference.
I doubt that the LLMs do this. It would be interesting to see whether they can actually relate the images to the coordinates—I suspect not i.e. I expect that they could draw a good map, with gridlines from training data but would be unable to relate the visual to the question. I expect that they are working from coordinates in wikipedia articles and the CIA website.
Another suggestion would be to ask the LLM to draw a map of the world with non-standard grid lines e.g. every 7 degrees
Really curious how humans would perform on this.
Humans would draw a map of the world from memory, overlay the grid and look up the reference. I doubt that the LLMs do this. It would be interesting to see whether they can actually relate the images to the coordinates—I suspect not i.e. I expect that they could draw a good map, with gridlines from training data but would be unable to relate the visual to the question. I expect that they are working from coordinates in wikipedia articles and the CIA website. Another suggestion would be to ask the LLM to draw a map of the world with non-standard grid lines e.g. every 7 degrees