Is this a typo/formatting bug? (Paragraph context: “Note that in our model architecture, the MLP’s hidden dimension is always scaled to stay smaller than the input vocabulary. n_input_vocab** ..”)
Yes, thanks. We wrote this in another editor and used Claude to port it to LW. Some formatting got screwed up in the transition. We fixed what we spotted, but there’s probably some issues remaining.
Is this a typo/formatting bug? (Paragraph context: “Note that in our model architecture, the MLP’s hidden dimension is always scaled to stay smaller than the input vocabulary. n_input_vocab** ..”)
Yes, thanks. We wrote this in another editor and used Claude to port it to LW. Some formatting got screwed up in the transition. We fixed what we spotted, but there’s probably some issues remaining.