I’m curious how well a model finetuned on the Alignment Newsletter performs at summarizing new content (probably blog posts; I’d assume papers are too long and rely too much on figures). My guess is that it doesn’t work very well even for blog posts, which is why I haven’t tried it yet, but I’d still be interested in the results and would love it on the off chance that it actually was good enough to save me some time.
We could definitely look into making the project evolve in this direction. In fact, we’re building a dataset of alignment-related texts and a small part of the dataset includes a scrape of arXiv papers extracted from the Alignment Newsletter. We’re working towards building GPT models fine-tuned on the texts.
I’m curious how well a model finetuned on the Alignment Newsletter performs at summarizing new content (probably blog posts; I’d assume papers are too long and rely too much on figures). My guess is that it doesn’t work very well even for blog posts, which is why I haven’t tried it yet, but I’d still be interested in the results and would love it on the off chance that it actually was good enough to save me some time.
We could definitely look into making the project evolve in this direction. In fact, we’re building a dataset of alignment-related texts and a small part of the dataset includes a scrape of arXiv papers extracted from the Alignment Newsletter. We’re working towards building GPT models fine-tuned on the texts.
Ya, I was even planning on trying:
Then feed that input to.
to see if that has some higher-quality summaries.
Well, one “correct” generalization there is to produce much longer summaries, which is not actually what we want.
(My actual prediction is that changing the karma makes very little difference to the summary that comes out.)