What is the role of Chat-GPT? Do you see it as progress over GPT-3, or is it just a tool for discovering capabilities that were already available in GPT-3 to good prompt engineers? [...] Is the importance that Chat is revealing those abilities and narrowing the ignorance?
Yes, it had revealed to me that GPT-3 was stronger than I had thought. I played with GPT-3 prior to ChatGPT, but it seems I was never very good at finding a good prompt. For example, I had tried to make it produce dialogue, in a similar manner to that of ChatGPT, but its replies were often surprisingly incoherent. On top of that, it would often produce boilerplate replies in the dialogue that were quite superficial, almost like the much worse BlenderBot from Meta.
After playing with ChatGPT however, and after seeing many impressive results on Twitter, I realized that the model’s fundamental capabilities were solidly on the right end of the distribution of what I had previously believed. I truly underestimated the power of getting the right prompt, or fine-tuning it. It was a stronger update than almost anything else I have seen from any language model.
What I get from essentially the same observations of ChatGPT is increase in AI risk without shortening of timelines, which were already with median at 2032-2042 for me. My model is that there is a single missing piece to the puzzle (of AGI, not alignment), generation of datasets for SSL (and then an IDA loop does the rest). This covers a current bottleneck, but also feels like a natural way of fixing the robustness woes.
Before ChatGPT, I expected that GPT-n is insufficiently coherent to set it up directly, in something like HCH bureaucracies, and fine-tuned versions tend to lose their map of the world, what they generate can no longer be straightforwardly reframed into an improvement over (amplification of) what the non-fine-tuned SSL phase trained on. This is good, because I expect a more principled method of filling the gaps in the datasets for SSL is the sort of reflection (in the usual human sense) that boosts natural abstraction, makes learning less lazy, promotes easier alignment. If straightforward bureaucracies of GPT-n can’t implement reflection, that is a motivation to figure out how to do this better.
But now I’m more worried that GPT-n with some fine-tuning and longer-term memory for model instances could be sufficiently close to human level to do reflection/generation directly, without a better algorithm. And that’s an alignment hazard, unless there is a stronger resolve to only use this for strawberry alignment tasks not too far away from human level of capability, which I’m not seeing at all.
I played with davinci, text-davinci-002, and text-davinci-003, if I recall correctly. The last model had only been out for a few days at most, however, before ChatGPT was released.
Of course, I didn’t play with any of these models in enough detail to become an expert prompt engineer. I mean, otherwise I would have made the update sooner
Yes, it had revealed to me that GPT-3 was stronger than I had thought. I played with GPT-3 prior to ChatGPT, but it seems I was never very good at finding a good prompt. For example, I had tried to make it produce dialogue, in a similar manner to that of ChatGPT, but its replies were often surprisingly incoherent. On top of that, it would often produce boilerplate replies in the dialogue that were quite superficial, almost like the much worse BlenderBot from Meta.
After playing with ChatGPT however, and after seeing many impressive results on Twitter, I realized that the model’s fundamental capabilities were solidly on the right end of the distribution of what I had previously believed. I truly underestimated the power of getting the right prompt, or fine-tuning it. It was a stronger update than almost anything else I have seen from any language model.
What I get from essentially the same observations of ChatGPT is increase in AI risk without shortening of timelines, which were already with median at 2032-2042 for me. My model is that there is a single missing piece to the puzzle (of AGI, not alignment), generation of datasets for SSL (and then an IDA loop does the rest). This covers a current bottleneck, but also feels like a natural way of fixing the robustness woes.
Before ChatGPT, I expected that GPT-n is insufficiently coherent to set it up directly, in something like HCH bureaucracies, and fine-tuned versions tend to lose their map of the world, what they generate can no longer be straightforwardly reframed into an improvement over (amplification of) what the non-fine-tuned SSL phase trained on. This is good, because I expect a more principled method of filling the gaps in the datasets for SSL is the sort of reflection (in the usual human sense) that boosts natural abstraction, makes learning less lazy, promotes easier alignment. If straightforward bureaucracies of GPT-n can’t implement reflection, that is a motivation to figure out how to do this better.
But now I’m more worried that GPT-n with some fine-tuning and longer-term memory for model instances could be sufficiently close to human level to do reflection/generation directly, without a better algorithm. And that’s an alignment hazard, unless there is a stronger resolve to only use this for strawberry alignment tasks not too far away from human level of capability, which I’m not seeing at all.
FWIW they call ChatGPT “GPT-3.5”, but text-davinci-002 was also in this series
Which model were you playing with before (davinci/text-davinci-002/code-davinci-002)?
I played with davinci, text-davinci-002, and text-davinci-003, if I recall correctly. The last model had only been out for a few days at most, however, before ChatGPT was released.
Of course, I didn’t play with any of these models in enough detail to become an expert prompt engineer. I mean, otherwise I would have made the update sooner