Dohun Lee

Karma: 47

Dohun Lee 25 Jul 2026 3:36 UTC
1 point
0
in reply to: Ziqian Zhong’s comment on: Data filtering works a lot worse than you would expect
Hi @Ziqian Zhong—sorry for the late reply! Unfortunately this is quite work from a few months ago with to plans to open source at the moment—however, I am happy to share with you the exact datasets/experiment details offline I used for the experiments, if that is useful—DM me!

Dohun Lee 8 Jul 2026 19:40 UTC
3 points
0
in reply to: draganover’s comment on: Data filtering works a lot worse than you would expect
We did take a quick look at phantom transfer at the start of this project, am personally more of the view that phantom transfer is caused by very subtle semantic effects. In the case of your example, I suspect that if we took a real catholicism expert, he/she would be able to identify that every document in the phantom transfer set slightly favors catholicism. Of course it could still be that the catholicism persona is getting up weighted of course!
With the case of SFT—I think your hypothesis is reasonable. I really liked the experiments in this write up, trying to isolate behaviors that are unique to some models and try differently distilled SFT datasets to see if they remain, like was done here: https://www.lesswrong.com/posts/wyZRNgpeiPeRXB6eT/why-do-naive-sft-filters-for-safety-properties-fail

Dohun Lee 8 Jul 2026 19:32 UTC
1 point
0
in reply to: Kyle O’Brien’s comment on: Data filtering works a lot worse than you would expect
Agree—this is why we re-ran some of the key experiments starting from the pre-train only version.

Data filtering works a lot worse than you would expect

Dohun Lee, J Rosser, Josh Engels and Neel Nanda

7 Jul 2026 4:41 UTC

59 points

9 comments3 min readLW link

Dohun Lee 18 May 2026 8:11 UTC
4 points
0
on: MATS 9 Retrospective & Advice
Great post! Thank you so much for writing it, I really enjoyed reading it. Some questions:
1. I would be interested to hear why you think the opportunity cost of not working on your project (vs applying to jobs/networking) is so high, given the MATS extension? Is it because of the relationship to your mentor? As someone coming to MATS from outside the Bay Area, would have thought its a great opportunity to make most of the vibrant community there. Would be interested to hear if you thought there were events you thought were particularly valuable/not valuable as well
1. On a related note, would be quite interested to hear what most fellows/yourself end up doing afterwards! Is the mode to go to London for the extension/does it differ significantly by background etc.