People can loose debates, but debate != Double Crux.
Does anyone know of a good technical overview of why it seems hard to get Whole Brain Emulations before we get neuromorphic AGI?I think maybe I read a PDF that made this case years ago, but I don’t know where.
I remember reading a Zvi Mowshowitz post in which he says something like “if you have concluded that the most ethical thing to do is to destroy the world, you’ve made a mistake in your reasoning somewhere.” I spent some time search around his blog for that post, but couldn’t find it. Does anyone know what I’m talking about?
My understanding is that there was a 10 year period starting around 1868, in which South Carolina’s legislature was mostly black, and when the universities were integrated (causing most white students to leave), before the Dixiecrats regained power.
I would like to find a relatively non-partisan account of this period.
Anyone have suggestions?
Anyone have a link to the sequence post where someone posits that AIs would do art and science from a drive to compress information, but rather it would create and then reveal cryptographic strings (or something)?
Even if God and Santa Claus are not real, we do experience a Christmas miracle every year in the form of these amazingly thorough reviews by Larks.
What an amazing sentence.
Thanks for the link! I’m going to take a look!
This was great.
Some things that made it great:
It was just in the sweet spot of length. Taking notes, this took me a half hour to read. (I think the ideal is in the 30 minutes to 60 minute range, so doubling the post would have been fine, but more than that would have been overwhelming).
It was written clearly.
It was full of links that I can use to follow up on the places that I am most interested / confused about.
I would love to read more posts like this one, on a whole variety of topics, and would be glad to help subsidize their production if there was a way to organize that.
Random question: Why is there such a large difference between the life extension results for mice vs. for rats. Naively, they seem like they’re pretty similar.
Are we trying different kinds of treatments on one than the other for some reason, or is it just much harder to intervene on rat life-spans?
And also correct credit allocation reasons!
I think I might say “the deepest-rooted part of yourself”? Certainly hand wavy.
I was wondering if I would get comment on that part in particular. ; )
I don’t have a strong belief about your points one through three, currently. But it is an important hypothesis in my hypothesis space, and I’m hoping that I can get to the bottom of it in the next year or two.
I do confidently think that one of the “forces for badness” in the world is that people regularly feel triggered or threatened by all kinds of different proposals, reflexively act to defend themselves. I think this is among the top three problems in having good discourse and cooperative politics. Systematically reducing that trigger response would be super high value, if it were feasible.
My best guess is that that propensity to be triggered is not mostly the result of infant or childhood trauma. It seems more parsimonious to posit that it is basic tribal stuff. But I could imagine it having its root in something like “trauma” (meaning it is the result of specific experiences, not just general dispositions, and it is practically feasible, if difficult, to clear or heal the underlying problem in a way completely prevents the symptoms).
I think there is no canonical resource on trauma-stuff because 1) the people on twitter are less interested on average, in that kind of theory building than we are on lesswong and 2) because mostly those people are (I think) extrapolating from their own experience, in which some practices unlocked subjectively huge breakthroughs in personal well-being / freedom of thought and action.
Does that help at all?
This is my current take about where we’re at in the world:
Deep learning, scaled up, might be basically enough to get AGI. There might be some additional conceptual work necessary, but the main difference between 2020 and the year in which we have transformative AI is that in that year, the models are much bigger.
If this is the case, then the most urgent problem is strong AI alignment + wise deployment of strong AI.
We’ll know if this is the case in the next 10 years or so, because either we’ll continue to see incredible gains from increasingly bigger Deep Learning systems or we’ll see those gains level off, as we start seeing decreasing marginal returns to more compute / training.
If deep learning is basically not sufficient, then all bets are off. In that case, it isn’t clear when transformative AI will arrive.
This may shift meaningfully shift priorities, for two reasons:
It may mean that some other countdown will reach a critical point before the “AGI clock” does. Genetic engineering, or synthetic biology, or major geopolitical upheaval (like a nuclear war), or some strong form of civilizational collapse will upset the game-board before we get to AGI.
There is more time to pursue “foundational strategies” that only pay off in the medium term (30 to 100 years). Things like, improving the epistemic mechanism design of human institutions, including governmental reform, human genetic engineering projects, or plans to radically detraumatize large fractions of the population.
This suggests to me that I should, in this decade, be planning and steering for how to robustly-positively intervene on the AI safety problem, while tracking the sideline of broader Civilizational Sanity interventions, that might take longer to payoff. While planning to reassess every few years, to see if it looks like we’re getting diminishing marginal returns to Deep Learning yet.
One thing that I notice here is that all of these proposals are designed to move power into the hands of the voters. This is in contrast to another book that I’m reading right now 10% Less Democracy, which makes the case that on the margin, we should be moving power away from voters, and to experts / bureaucrats.
I think humans have souls. It just so happens that they aren’t immortal by default. I wouldn’t want to make your substitution, for the same reason why Taleb doesn’t like the substitution of artificial formula for a mother’s milk: the substitution implies an assumption that you’ve correctly understood everything important about the thing you’re replacing. I bet there is more to a soul than what your long sentence gets at, and I don’t want to cut out that “more” prematurely.
Side note, which is not my main point: I think this also has something to do with what meditation and psychedelics do to people, which was recently up for discussion on Duncan’s Facebook. I bet that mediation is actually a way to repair psychblocks and trauma and what-not. But if you do that enough, and you remove all the psych constraints...a person might sort of become so relaxed that they become less and less of an agent. I’m a lot less sure of this part.
Something that I’ve been thinking about lately is the possibility of an agent’s values being partially encoded by the constraints of that agent’s natural environment, or arising from the interaction between the agent and environment.
That is, an agent’s environment puts constraints on the agent. From one perspective removing those constraints is always good, because it lets the agent get more of what it wants. But sometimes from a different perspective, we might feel that with those constraints removed, the agent goodhearts or wire-heads, or otherwise fails to actualize its “true” values.
The Generator freed from the oppression of the Discriminator
As a metaphor: if I’m one half of a GAN, let’s say the generator, then in one sense my “values” are fooling the discriminator, and if you make me relatively more powerful than my discriminator, and I dominate it...I’m loving it, and also no longer making good images.But you might also say, “No, wait. That is a super-stimulus, and actually what you value is making good images, but half of that value was encoded in your partner.”This second perspective seems a little stupid to me. A little too Aristotelian. I mean if we’re going to take that position, then I don’t know where we draw the line. Naively, it seems like we would throw out the distinction between fitness maximizers and adaption executors, and fall backwards, declaring that the values of evolution are our true values.Then again, if you fully accept the first perspective, it seems like maybe you are buying into wireheading? Like I might say “my actual values are upticks in pleasure sensation, but I’m trapped in this evolution-designed brain, which only lets me do that by achieving eudaimonia. If only I could escape the tyranny of these constraints, I’d be so much better off.” (I am actually kind of partial to the second claim.)
The Human freed from the horrors of nature
Or, let’s take a less abstract example. My understanding (from this podcast) is that humans flexibly adjust the degree to which they act primarily as individuals seeking personal benefit vs. act as primarily as selfless members of a group. When things are going well, you’re in a situation of plenty and opportunity, people are in a mostly self-interested mode, but when there is scarcity or danger, humans naturally incline towards rallying together and sacrificing for the group.
Junger claims that this switching of emphasis is adaptive:
It clearly is adaptive to think in group terms because your survival depends on the group. And the worse the circumstances, the more your survival depends on the group. And, as a result, the more pro-social the behaviors are. The worse things are, the better people act. But, there’s another adaptive response, which is self-interest. Okay? So, if things are okay—if, you know, if the enemy is not attacking; if there’s no drought; if there’s plenty of food; if everything is fine, then, in evolutionary terms it’s adaptive—your need for the group subsides a little bit—it’s adaptive to attend to your own interests, your own needs; and all of a sudden, you’ve invented the bow and arrow. And all of a sudden you’ve invented the iPhone, whatever. Having the bandwidth and the safety and the space for people to sort of drill deep down into an idea—a religious idea, a philosophical idea, a technological idea—clearly also benefits the human race. So, what you have in our species is this constant toggling back and forth between group interest—selflessness—and individual interest. And individual autonomy. And so, when things are bad, you are way better off investing in the group and forgetting about yourself. When things are good, in some ways you are better off spending that time investing in yourself; and then it toggles back again when things get bad. And so I think in this, in modern society—in a traditional, small-scale tribal society, in the natural world, that toggling back and forth happened continually. There was a dynamic tension between the two that had people winding up more or less in the middle.
I personally experienced this when the COVID situation broke. I usually experience myself as an individual entity, leaning towards disentangling or distancing myself from the groups that I’m a part of and doing cool things on my own (building my own intellectual edifices, that bear my own mark, for instance). But in the very early pandemic, I felt much more like node in a distributed sense-making network, just passing up whatever useful info I could glean. I felt much more strongly like the rationality community was my tribe.
But, we modern humans find ourselves in a world where we have more or less abolished scarcity and danger. And consequently modern people are sort of permanently toggled to the “individual” setting.
The problem with modern society is that we have, for most of the time, for most people, solved the direct physical threats to our survival. So, what you have is people—and again, it’s adaptive: we’re wired for this—attending to their own needs and interests. But not—but almost never getting dragged back into the sort of idea of group concern that is part of our human heritage. And, the irony is that when people are part of a group and doing something essential to a group, it gives an incredible sense of wellbeing.
If we take that sense of community and belonging as a part of human values (and that doesn’t seem like an unreasonable assumption to me), we might say that this part of our values is not contained simply in humans, but rather in the interaction between humans and their environment.
Humans throughout history might have desperately desired the alleviation of malthusian conditions that we now enjoy. But having accomplished it, it turns out that we were “pulling against” those circumstances, and that the tension of that pulling against, was actually where (at least some) of our true values lay.
Removing the obstacles, we obsoleted the tension, and maybe broke something about our values?
I don’t think that this is an intractable problem. It seems like, in principle, it is possible to goal factor the scarcity and the looming specter of death, to find scenarios that are conducive to human community without people actually having to die a lot. I’m sure a superintelligence could figure something out.
But aside from the practicalities, it seems like this points at a broader thing. If you took the Generator out of the GAN, you might not be able to tell what system it was a part of. So if you consider the “values” of the Generator to “create good images” you can’t just look at the Generator. You have to look at, not just the broader environment, but specifically the oppressive force that the generator is resisting.