Really like this direction, and excited that it’s finally becoming (more) mainstream but I disagree with the framing here on two points:
GAs as digital twins: It would be great to have some degree of transfer in values / context / thinking styles to my personal GA, but I also think this undervalues complementarity between humans and LLMs. The nice thing about personally tuned AI models is that you can reinforce the human + AI loops, which drives differentiation to some extent. The human does the things that the human is good at (e.g. out of distribution / novel situations, domain-specific knowledge, overall direction-setting) and the AI system does the things that the AI is good at (e.g fast inference within distribution, general knowledge). You can think of the AI system as amortising certain tasks that humans do frequently, leaving them to explore new parts of the distribution. The post itself does mention this: “Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.”, but I think a simple imitation objective cuts against amplification. Work on assistance games from Stuart Russell’s lab seems relevant here.
Project / Community GAs: The GA framing feels centered around this idea of “one model per person”, but if you’re doing dynamic fine-tuning, why not go even more fine-grained? Why not have a fork of your GA tuned specifically for when you’re at work (or multiple for different projects) and one for your personal life? And equally, you can go broader—you can have a model aligned with your friends or community, or organisation—or a particular mix of these, which you can then fork for your individual purposes (or weight the data mix by similarity to you), and get some elegant recursive properties.
Really like this direction, and excited that it’s finally becoming (more) mainstream but I disagree with the framing here on two points:
GAs as digital twins:
It would be great to have some degree of transfer in values / context / thinking styles to my personal GA, but I also think this undervalues complementarity between humans and LLMs. The nice thing about personally tuned AI models is that you can reinforce the human + AI loops, which drives differentiation to some extent. The human does the things that the human is good at (e.g. out of distribution / novel situations, domain-specific knowledge, overall direction-setting) and the AI system does the things that the AI is good at (e.g fast inference within distribution, general knowledge). You can think of the AI system as amortising certain tasks that humans do frequently, leaving them to explore new parts of the distribution. The post itself does mention this: “Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.”, but I think a simple imitation objective cuts against amplification. Work on assistance games from Stuart Russell’s lab seems relevant here.
Project / Community GAs:
The GA framing feels centered around this idea of “one model per person”, but if you’re doing dynamic fine-tuning, why not go even more fine-grained? Why not have a fork of your GA tuned specifically for when you’re at work (or multiple for different projects) and one for your personal life? And equally, you can go broader—you can have a model aligned with your friends or community, or organisation—or a particular mix of these, which you can then fork for your individual purposes (or weight the data mix by similarity to you), and get some elegant recursive properties.