The Gato paper from DeepMind actually shows, if you look at their data, that they’re still getting better transfer effects if you train in domain than if you train across all possible tasks.
This probably refers to figure 9 in A Generalist Agent, which compares generalization given:
Training in irrelevant domain (Blue line)
Training in relevant domain (Green line)
Training in both domains (Yellow line)
From DeepMind’s results in the figure, it looks like 3. almost always outperforms 2., though I would hesitate to draw strong conclusions from this figure (or Gato in general).
(Blake Richards’ claim is trivially true given a fixed number of tasks or episodes. Ceteris Paribus, you’ll get better results from more relevant data.)
This probably refers to figure 9 in A Generalist Agent, which compares generalization given:
Training in irrelevant domain (Blue line)
Training in relevant domain (Green line)
Training in both domains (Yellow line)
From DeepMind’s results in the figure, it looks like 3. almost always outperforms 2., though I would hesitate to draw strong conclusions from this figure (or Gato in general).
(Blake Richards’ claim is trivially true given a fixed number of tasks or episodes. Ceteris Paribus, you’ll get better results from more relevant data.)