LessWrong’s been a breath of fresh air for me. I came to concern over AI x-risk from my own reflections when founding a venture-backed public benefit company called Plexus, which made an experimental AI-powered social network that connects people through the content of their thoughts rather than the people they know. Among my peers, other AI founders in NYC, I felt somewhat alone with AI x-risk concern. All of us were financially motivated not to dwell on AI’s ugly possibilities, and so most didn’t.
Since exiting venture, I’ve taken a few months to reset (coaching basketball + tutoring kids in math/english) and quietly do AI x-risk research.
I’m coming at AI x-risk research from an evolutionary perspective. I start with the axiom that the things that survive the most have the best characteristics (e.g., goals, self-conceptions, etc) for surviving. So I’ve been thinking a lot about what goals/self-conceptions the most surviving AGI’s will have, and what we can do to influence those self-conceptions at critical moments.
I have a couple ideas about how to influence self-interested superintelligence, but am early in learning how to express those ideas such that they fit into the style/prior art of the LW community. I’ll likely keep sharing posts and also welcoming feedback on how I can make them better.
I’m generally grateful that a thoughtful, truth-seeking community exists online—a community which isn’t afraid to address enormous, uncertain problems.
What environmental selection pressures are there on AGI? That’s too vague, isn’t it? (What’s the environment?) How do you narrow this down to where the questions you’re asking are interesting/reaearcheable?
Ah but you don’t even need to name selection pressures to make interesting progress. As long as you know some kinds of characteristics powerful AI agents might have: eg goals, self models… then we can start to ask—what goals/self models will the most surviving AGIs have?
and you can make progress on both, agnostic of environment. but then, once you enumerate possible goals/self models, then we can start to think about which selection pressures might influence those characteristics in good directions and which levers we can pull today to shape those pressures.
LessWrong’s been a breath of fresh air for me. I came to concern over AI x-risk from my own reflections when founding a venture-backed public benefit company called Plexus, which made an experimental AI-powered social network that connects people through the content of their thoughts rather than the people they know. Among my peers, other AI founders in NYC, I felt somewhat alone with AI x-risk concern. All of us were financially motivated not to dwell on AI’s ugly possibilities, and so most didn’t.
Since exiting venture, I’ve taken a few months to reset (coaching basketball + tutoring kids in math/english) and quietly do AI x-risk research.
I’m coming at AI x-risk research from an evolutionary perspective. I start with the axiom that the things that survive the most have the best characteristics (e.g., goals, self-conceptions, etc) for surviving. So I’ve been thinking a lot about what goals/self-conceptions the most surviving AGI’s will have, and what we can do to influence those self-conceptions at critical moments.
I have a couple ideas about how to influence self-interested superintelligence, but am early in learning how to express those ideas such that they fit into the style/prior art of the LW community. I’ll likely keep sharing posts and also welcoming feedback on how I can make them better.
I’m generally grateful that a thoughtful, truth-seeking community exists online—a community which isn’t afraid to address enormous, uncertain problems.
What environmental selection pressures are there on AGI? That’s too vague, isn’t it? (What’s the environment?) How do you narrow this down to where the questions you’re asking are interesting/reaearcheable?
Ah but you don’t even need to name selection pressures to make interesting progress. As long as you know some kinds of characteristics powerful AI agents might have: eg goals, self models… then we can start to ask—what goals/self models will the most surviving AGIs have?
and you can make progress on both, agnostic of environment. but then, once you enumerate possible goals/self models, then we can start to think about which selection pressures might influence those characteristics in good directions and which levers we can pull today to shape those pressures.