The models discuss the paradox of diversity in cultural evolution and how specialization affects cultural complexity and innovation rates in societies. Diversity fuels innovation through recombination but also divides people.
Social learning is most effective when the environment is moderately variable, not too stable or unstable.
Larger population sizes and connectivity enable higher cultural complexity and innovation through a “collective brain” effect, but diversity also creates inequality.
There is a trade-off between diversity, which enables more innovation potential, and coordination and communication, which diversity hinders.
As cultural domains become more complex, larger effective population sizes are needed to maintain skill levels due to the knowledge that needs to be transmitted.
There are strategies to deal with the paradox of diversity, like using translators and partially acculturated populations.
Cooperation enables larger scales of collective action but is also undermined by lower scales of cooperation, like when nepotism undermines institutions.
The availability of resources and energy affects the scale of cooperation, enabling larger collective efforts when more abundant.
Abundance enables a “collective brain” mindset while scarcity fosters a zero-sum, competitive psychology.
Punctuated rises in cooperation may occur when new levels of resources unlock higher scales of collective action.
Cultural evolution can be viewed as evolution applied to the substrate of culture, where memes (ideas, behaviors, information) are replicated and selected.
Cultural evolution enables the rapid acquisition and improvement of skills across an expanding range of tasks, which could help generate artificial general intelligence.
Cultural transmission, especially real-time cultural transmission, is difficult but important for building cultural evolution-aware cooperative AI and potentially improving safety.
Cooperation is necessary for AI, as single agents are not enough to create enough pressure for complex social interactions.
Cultural evolution may help ratchet up abilities like theory of mind and self-domestication that enable cooperation.
Some level of general intelligence may be needed to kickstart cumulative cultural evolution, though AI may currently have enough capabilities.
Cultural evolution provides a unique angle to identify potential problems early through its inherently interactive nature.
The setup discussed focuses on a fully cooperative environment where all agents share the same goal.
Continual learning and lifelong learning techniques may help address issues of “catastrophic forgetting” in cultural transmission models.
While cultural evolution may not be the only way to generate safe general intelligence, it could still provide useful inspiration.
Gillian Hadfield discusses the importance of cooperative intelligence and normative systems for AI. She argues that humans have evolved the ability to create and enforce norms through third-party punishment, which allows for stable groups and cooperation. However, current AI approaches focus too much on individual optimization. Instead, AI systems should learn to participate in and maintain normative infrastructure, rather than simply mimic existing human behavior. Understanding the generative process behind human norms and the role of normative reasoning may help build more cooperative AI systems. Silly rules, though seemingly unimportant, can serve as signals of group compliance and help maintain group stability.
Cooperative intelligence is fundamental to human intelligence. It’s not just about task completion and optimization, but also the capacity for cooperation with others.
The most fundamental form of cooperation humans engage in is creating and maintaining the normative infrastructure of cooperative groups through norms and enforcement.
Third party enforcement expands the set of possible solutions for cooperation because almost any equilibrium can be achieved if the group is coordinated to enforce norms.
Silly rules, or rules with no direct impact on welfare, can help stabilize groups by signaling willingness to comply with and enforce important rules.
The plasticity and capacity for changing content while remaining stable makes normativity a valuable tool.
Building AI that can participate in and be competent actors within normative infrastructure is more complex than just stuffing norms into them.
AI should observe how uncertainty about punishable actions is resolved and where decision making around norms comes from.
Normativity in humans involves giving reasons and assessing what constitutes good reasons, which is itself subject to normative structure.
Internal moral reasoning can represent the group’s evaluation of one’s behavior and predict third party enforcement.
The model of the utility maximizing selfish individual is not representative of how actual humans in groups behave.
Evolutionary game theory studies how strategies evolve and change over time, unlike classical game theory which focuses on static strategies. Natural selection, not rational choice, drives the evolution of strategies in biological systems. Initially, defectors outcompete cooperators but repeated interactions allow cooperation to evolve through strategies like tit-for-tat and generous tit-for-tat. Indirect reciprocity through reputation systems also enables cooperation in larger groups where people do not interact repeatedly. The evolution of cooperation through reciprocity, reputation, and social norms is a defining feature of human societies.
Evolutionary game theory focuses on the dynamics of strategy change over time and how strategies evolve, whereas classical game theory focuses on static strategies.
In evolutionary game theory, players do not act rationally but strategies that survive over time are considered optimal.
Defection is often the optimal strategy in a single-shot game, but cooperation can evolve in repeated games through punishment of non-cooperation and reward of cooperation.
The tit for tat strategy, which is cooperative but also quick to retaliate against defectors, is often successful in repeated prisoner’s dilemma games.
Generous tit for tat, a more forgiving version of tit for tat, can be an evolutionary stable strategy in noisy environments.
Indirect reciprocity through reputation systems can enable cooperation in large societies where people only interact once.
Reputation systems favor cooperators who then have more opportunities and success.
Humans have mastered indirect reciprocity and developed social intelligence and institutions to a greater extent than other species.
Norms, guilt, and shame help enforce cooperation and good behavior in groups.
Socio-cultural institutions enable advanced forms of human cooperation.
Survival of the fittest does not preclude altruism in nature. Simulations show that unconditionally sacrificing offspring for others does not work in the long run. For altruism to evolve, there must be some benefit to copies of the altruistic gene. Kin selection, where creatures help family members who likely share the same genes, can allow altruistic genes to spread through a population if the benefit of helping outweighs the cost. While the genes are selfish in seeking to replicate, the altruistic behavior of the creatures themselves is genuine.
Survival of the fittest does not mean that creatures cannot act altruistically by hurting their own chances to help others.
Unconditional altruism is not a successful long term strategy as it helps competitors as much as itself.
For altruism to be successful, the cost of the altruistic act needs to be lower than the benefit it provides.
Altruism towards all indiscriminately is rare in nature.
Altruism towards those with a detectable trait like a “green beard” can allow altruistic creatures to coordinate and benefit each other.
Traits for altruism and detectable traits tend to become separated over time, breaking the coordination.
Kin altruism towards family members can be successful as family are likely to share the same altruism gene.
For kin altruism to work, the benefit of the altruistic act needs to outweigh the cost, on average.
The genes involved in altruism are still selfish—they just coordinate copies of themselves.
The creatures themselves can genuinely act altruistically despite their genes being selfish.
evo gt, morality, altruism, etc
The models discuss the paradox of diversity in cultural evolution and how specialization affects cultural complexity and innovation rates in societies. Diversity fuels innovation through recombination but also divides people.
Social learning is most effective when the environment is moderately variable, not too stable or unstable.
Larger population sizes and connectivity enable higher cultural complexity and innovation through a “collective brain” effect, but diversity also creates inequality.
There is a trade-off between diversity, which enables more innovation potential, and coordination and communication, which diversity hinders.
As cultural domains become more complex, larger effective population sizes are needed to maintain skill levels due to the knowledge that needs to be transmitted.
There are strategies to deal with the paradox of diversity, like using translators and partially acculturated populations.
Cooperation enables larger scales of collective action but is also undermined by lower scales of cooperation, like when nepotism undermines institutions.
The availability of resources and energy affects the scale of cooperation, enabling larger collective efforts when more abundant.
Abundance enables a “collective brain” mindset while scarcity fosters a zero-sum, competitive psychology.
Punctuated rises in cooperation may occur when new levels of resources unlock higher scales of collective action.
https://www.youtube.com/watch?v=oqV23pC4mhA
Cultural evolution can be viewed as evolution applied to the substrate of culture, where memes (ideas, behaviors, information) are replicated and selected.
Cultural evolution enables the rapid acquisition and improvement of skills across an expanding range of tasks, which could help generate artificial general intelligence.
Cultural transmission, especially real-time cultural transmission, is difficult but important for building cultural evolution-aware cooperative AI and potentially improving safety.
Cooperation is necessary for AI, as single agents are not enough to create enough pressure for complex social interactions.
Cultural evolution may help ratchet up abilities like theory of mind and self-domestication that enable cooperation.
Some level of general intelligence may be needed to kickstart cumulative cultural evolution, though AI may currently have enough capabilities.
Cultural evolution provides a unique angle to identify potential problems early through its inherently interactive nature.
The setup discussed focuses on a fully cooperative environment where all agents share the same goal.
Continual learning and lifelong learning techniques may help address issues of “catastrophic forgetting” in cultural transmission models.
While cultural evolution may not be the only way to generate safe general intelligence, it could still provide useful inspiration.
https://www.youtube.com/watch?v=9oxOcKrCmBk
Gillian Hadfield discusses the importance of cooperative intelligence and normative systems for AI. She argues that humans have evolved the ability to create and enforce norms through third-party punishment, which allows for stable groups and cooperation. However, current AI approaches focus too much on individual optimization. Instead, AI systems should learn to participate in and maintain normative infrastructure, rather than simply mimic existing human behavior. Understanding the generative process behind human norms and the role of normative reasoning may help build more cooperative AI systems. Silly rules, though seemingly unimportant, can serve as signals of group compliance and help maintain group stability.
Cooperative intelligence is fundamental to human intelligence. It’s not just about task completion and optimization, but also the capacity for cooperation with others.
The most fundamental form of cooperation humans engage in is creating and maintaining the normative infrastructure of cooperative groups through norms and enforcement.
Third party enforcement expands the set of possible solutions for cooperation because almost any equilibrium can be achieved if the group is coordinated to enforce norms.
Silly rules, or rules with no direct impact on welfare, can help stabilize groups by signaling willingness to comply with and enforce important rules.
The plasticity and capacity for changing content while remaining stable makes normativity a valuable tool.
Building AI that can participate in and be competent actors within normative infrastructure is more complex than just stuffing norms into them.
AI should observe how uncertainty about punishable actions is resolved and where decision making around norms comes from.
Normativity in humans involves giving reasons and assessing what constitutes good reasons, which is itself subject to normative structure.
Internal moral reasoning can represent the group’s evaluation of one’s behavior and predict third party enforcement.
The model of the utility maximizing selfish individual is not representative of how actual humans in groups behave.
https://www.youtube.com/watch?v=BCQJ2G3_Hn4
Evolutionary game theory studies how strategies evolve and change over time, unlike classical game theory which focuses on static strategies. Natural selection, not rational choice, drives the evolution of strategies in biological systems. Initially, defectors outcompete cooperators but repeated interactions allow cooperation to evolve through strategies like tit-for-tat and generous tit-for-tat. Indirect reciprocity through reputation systems also enables cooperation in larger groups where people do not interact repeatedly. The evolution of cooperation through reciprocity, reputation, and social norms is a defining feature of human societies.
Evolutionary game theory focuses on the dynamics of strategy change over time and how strategies evolve, whereas classical game theory focuses on static strategies.
In evolutionary game theory, players do not act rationally but strategies that survive over time are considered optimal.
Defection is often the optimal strategy in a single-shot game, but cooperation can evolve in repeated games through punishment of non-cooperation and reward of cooperation.
The tit for tat strategy, which is cooperative but also quick to retaliate against defectors, is often successful in repeated prisoner’s dilemma games.
Generous tit for tat, a more forgiving version of tit for tat, can be an evolutionary stable strategy in noisy environments.
Indirect reciprocity through reputation systems can enable cooperation in large societies where people only interact once.
Reputation systems favor cooperators who then have more opportunities and success.
Humans have mastered indirect reciprocity and developed social intelligence and institutions to a greater extent than other species.
Norms, guilt, and shame help enforce cooperation and good behavior in groups.
Socio-cultural institutions enable advanced forms of human cooperation.
https://www.youtube.com/watch?v=HxgVYhhArSk
Survival of the fittest does not preclude altruism in nature. Simulations show that unconditionally sacrificing offspring for others does not work in the long run. For altruism to evolve, there must be some benefit to copies of the altruistic gene. Kin selection, where creatures help family members who likely share the same genes, can allow altruistic genes to spread through a population if the benefit of helping outweighs the cost. While the genes are selfish in seeking to replicate, the altruistic behavior of the creatures themselves is genuine.
Survival of the fittest does not mean that creatures cannot act altruistically by hurting their own chances to help others.
Unconditional altruism is not a successful long term strategy as it helps competitors as much as itself.
For altruism to be successful, the cost of the altruistic act needs to be lower than the benefit it provides.
Altruism towards all indiscriminately is rare in nature.
Altruism towards those with a detectable trait like a “green beard” can allow altruistic creatures to coordinate and benefit each other.
Traits for altruism and detectable traits tend to become separated over time, breaking the coordination.
Kin altruism towards family members can be successful as family are likely to share the same altruism gene.
For kin altruism to work, the benefit of the altruistic act needs to outweigh the cost, on average.
The genes involved in altruism are still selfish—they just coordinate copies of themselves.
The creatures themselves can genuinely act altruistically despite their genes being selfish.
https://www.youtube.com/watch?v=lFEgohhfxOA