oekenta

• oekenta 1 Sep 2020 19:51 UTC

  LW: 2 AF: 1

  AF

  in reply to: nostalgebraist’s comment on: in­ter­pret­ing GPT: the logit lens

  Thanks for the info.

  This was a great read, very informative.

• oekenta 1 Sep 2020 18:56 UTC

  3 points

  in reply to: Gurkenglas’s comment on: in­ter­pret­ing GPT: the logit lens

  I think I understand your question and was also confused by this for a bit so I wanted add in some points of clarification. First I want out that I really couldn’t find a satisfactory explanation of this particular detail (at least one that I could understand) so I pieced this together myself from looking at the huggingface code for GPT2. I may get some details wrong.

  During training at each step the GPT2 takes in an N tokens and outputs N tokens. But the i-th output token is computed in such away that it only relies on the information from tokens 1, …, i and is meant to predict i+1-th token from these. I think it’s best to think of each output being computed independently of the others (though this isn’t strictly true since the separate outputs are computed by shared matrices). So for each i, we train the network so that the i-th output produces the correct result given the _input_ tokens 1, …, i. There is a term in the loss function for each output token and the total loss is the sum of all the losses of the output tokens. The outputs at other positions do not play a role in the i-th output token, only the first 1,..., i input tokens do.

  During inference, given an input of k tokens, we are only concerned with the k-th output token (which should predict the token following the first k). GPT-3 also produces predictions for the outputs before position k but these are just ignored since we already know what these values should be.

• oekenta 1 Sep 2020 18:07 UTC

  LW: 1 AF: 1

  AF

  on: in­ter­pret­ing GPT: the logit lens

  Hey I’m not finished reading this yet but I noticed something off about what you said.

  At the end, the final 1600-dimensional vector is multiplied by W’s transpose to project back into vocab space.

  
  

  This isn’t quite right. They don’t multiply by W’s transpose at the end. Rather there is a completely new matrix at the end, whose shape is the same as the transpose of W.

  You can see this in huggingface’s code for GPT2. In the class GPT2LMHeadModel the final matrix multiplication is performed by the matrix called “lm_head”, where as the matrix you call W which is used to map 50,257 dimensional vectors into 1600 dimensional space is called “wte” (found in the GPT2Model class). You can see from the code that wte has shape “Vocab size x Embed Size” while lm_head has shape “Embed Size x Vocab size” so lm_head does have the same shape as W transpose but doesn’t have the same numbers.

  
  

  Edit: I could be wrong here, though. Maybe lm_head was set to be equal to wte transpose? I’m looking through the GPT-2 paper but don’t see anything like that mentioned.