Taywon Min

Sy­co­phancy Towards Re­searchers Drives Perfor­ma­tive Misalignment

Trans­mit­ting Misal­ign­ment with Sublimi­nal Learn­ing via Paraphrasing

• Taywon Min 25 Jun 2025 5:15 UTC

  1 point

  0

  on: Greedy-Ad­van­tage-Aware RLHF

  Is it ok to compute the advantage function as a difference of Value functions?
  To my understanding, the advantage in ppo is not simply the difference between value functions, but uses GAE, which depends on later value, reward terms of the sampled trajectory.
  Shouldn’t we necessarily use that function during PPO training?

• Taywon Min 25 May 2025 7:56 UTC

  2 points

  0

  in reply to: ZY’s comment on: Emer­gent Misal­ign­ment: Nar­row fine­tun­ing can pro­duce broadly mis­al­igned LLMs

  But what if all insecure code contribute in some way?
  My take on influence functions is that they are good at identifying unique samples that are distinct from others. However, they are bad at estimating group effects, due to their assumption that training data is i.i.d.
  

  Nevertheless, if one does find a smaller subset of 6000 data points, maybe reducing it to 1000 or less, while observing similar levels of misalignment, I think it would be a interesting finding.

• Taywon Min 24 Feb 2025 11:30 UTC

  1 point

  0

  on: An X-Ray is Worth 15 Fea­tures: Sparse Au­toen­coders for In­ter­pretable Ra­diol­ogy Re­port Generation

  Thanks for the great work. I think that multimodal sparse auto encoders is a promising direction. Do you think it is possible /​ worthwhile to train SAEs on vla models like OpenVLA? I haven’t seen any related work training or interpreting action models using SAE work, and am curious of your thoughts.