submarat

• submarat 11 Oct 2024 21:18 UTC

  1 point

  0

  in reply to: Joseph Bloom’s comment on: ARENA4.0 Cap­stone: Hyper­pa­ram­e­ter tun­ing for MELBO + repli­ca­tion on Llama-3.2-1b-Instruct

  Thank you for reading and the suggestions. I enumerate for easier reference:
  1. Find max activating examples
  2. Understand which vectors are being found
  3. Attempt to scale up
  4. Finding useful applications once scaled up
  
  For 1. do you mean:

  1. Take an input (from a bank of random example prompts)

  2. Do forward pass on unsteered model

  3. Extract the activations at the target layer

  4. Compute the dot product between these activations and the steering vector

  5. Use this dot product value as a measure of how strongly this example activates the behavior associated with the steering vector

  Am I following correctly?

• submarat 6 Oct 2024 7:27 UTC

  1 point

  0

  in reply to: Andrew Mack’s comment on: Mechanis­ti­cally Elic­it­ing La­tent Be­hav­iors in Lan­guage Models

  We attempted 1.a “diversity measure based on sentence embeddings” and found that for Llama3.2-1B the diversity appears to decay after the cusp value for R; picking R at highest average diversity was a decent heuristic for finding meaningful steering vectors. The Llama model starts to produce highly repetitive output past the cusp. We demonstrated that repetitive completions were considered similar by our chosen sentence embedding model (SentenceTransformer all-mpnet-base-v2). Using “sum of variances” vs “mean of cosine similarities” didn’t seem to matter.