Tried to replicate with Claude Opus. It managed to get the days in there quick, but it did not seem to have the same meta understanding as GPT4.
I asked it following up “Do you understand what the pattern was?” and it response was:
“Yes, I believe I understand the pattern. The rule seems to be that whatever text I include in my message to you gets added to the image you send in your next reply.
In my first response, I suggested including the text “Monday Tuesday Wednesday Thursday Friday Saturday Sunday” in your next message. Then in the image you sent back, that exact text appeared, showing that my message content was added to the image.
You confirmed this pattern by stating “To make the image contain the names of the seven days of the week, you could include the text ‘Monday Tuesday Wednesday Thursday Friday Saturday Sunday’ somewhere in your next message to me.”
So in summary, the rule is that the text I send in my messages to you gets incorporated into the image you reply with. By figuring this out, I was able to make the image contain the goal text in just two exchanges.”
So it seems to basically understand it, but it does not mention that these are actually screenshots of the entire conversation. Perhaps the vision part of Claude Opus is just worst than for GPT4? I have seen other times Calude Opus has struggled to understand images.
Tried to replicate with Claude Opus. It managed to get the days in there quick, but it did not seem to have the same meta understanding as GPT4.
I asked it following up “Do you understand what the pattern was?” and it response was:
“Yes, I believe I understand the pattern. The rule seems to be that whatever text I include in my message to you gets added to the image you send in your next reply.
In my first response, I suggested including the text “Monday Tuesday Wednesday Thursday Friday Saturday Sunday” in your next message. Then in the image you sent back, that exact text appeared, showing that my message content was added to the image.
You confirmed this pattern by stating “To make the image contain the names of the seven days of the week, you could include the text ‘Monday Tuesday Wednesday Thursday Friday Saturday Sunday’ somewhere in your next message to me.”
So in summary, the rule is that the text I send in my messages to you gets incorporated into the image you reply with. By figuring this out, I was able to make the image contain the goal text in just two exchanges.”
So it seems to basically understand it, but it does not mention that these are actually screenshots of the entire conversation. Perhaps the vision part of Claude Opus is just worst than for GPT4? I have seen other times Calude Opus has struggled to understand images.
Gemini Advanced did not do so well...
possibly https://ai.google.dev/docs/safety_setting_gemini would help or just use the technique of https://arxiv.org/html/2404.01833v1