That doesn’t appear to be explained specifically, but what I think they are giving is the larger model size equivalence. That is to say, the 350M parameter language model with Mind’s Eye is about as good as a 2.5B parameter language model, and so on.
I might be missing something but are they not just giving the number of parameters (in millions of parameters) on a log10 scale? Scaling laws are usually by log-parameters, and I suppose they felt that it was cleaner to subtract the constant log(10^6) from all the results (e.g. taking log(1300) instead of log(1.3B)).
Interesting. Figure 2 looks pretty impressive. What do the phrases like “log(350) = 2.5B” at the bottom of the figure mean?
That doesn’t appear to be explained specifically, but what I think they are giving is the larger model size equivalence. That is to say, the 350M parameter language model with Mind’s Eye is about as good as a 2.5B parameter language model, and so on.
I might be missing something but are they not just giving the number of parameters (in millions of parameters) on a log10 scale? Scaling laws are usually by log-parameters, and I suppose they felt that it was cleaner to subtract the constant log(10^6) from all the results (e.g. taking log(1300) instead of log(1.3B)).
The B they put at the end is a bit weird though.