What’s the hoped-for benefit of this plan? I think this post would have been a lot better if you explained what the plan is supposed to do and why you think it can work.
Imagine you have a small corpus of data from good ethical people, and a large corpus of data that’s full of 4chan trolls or reward hacking or some other bad thing. (The large corpus isn’t just trolls, there is useful data in there too)
The hoped for benefit is that the AI displays all the useful knowledge that can be extracted from the large corpus, but only displays the behavior patterns found in the small corpus.
So suppose the 4chan data contains a few useful maths ideas, and also a lot of slurs. And suppose the good data consists entirely of maths proofs. Then the LLM can store proofs from both sources in the same common format. And “non-rot13 text is never a slur” is an easy to learn rule.
What’s the hoped-for benefit of this plan? I think this post would have been a lot better if you explained what the plan is supposed to do and why you think it can work.
Imagine you have a small corpus of data from good ethical people, and a large corpus of data that’s full of 4chan trolls or reward hacking or some other bad thing. (The large corpus isn’t just trolls, there is useful data in there too)
The hoped for benefit is that the AI displays all the useful knowledge that can be extracted from the large corpus, but only displays the behavior patterns found in the small corpus.
So suppose the 4chan data contains a few useful maths ideas, and also a lot of slurs. And suppose the good data consists entirely of maths proofs. Then the LLM can store proofs from both sources in the same common format. And “non-rot13 text is never a slur” is an easy to learn rule.
This links in with this post. https://www.lesswrong.com/posts/AfoGGrJfuNzofpzWL/models-may-behave-differently-in-graded-episodes-a-tirade Which says, amongst other things, that current models already mostly know whether they are in a graded episode or not, and this is good.