Questions and comments about Eliezer’s Dec. 2 2013 Oxford speech
Notes I took while listening to the speech:
Eliezer Yudkowsky on Friendly AI
If the human race is down to 1000 people, what are the odds that it will continue and do well? I realize this is a nitpick—the argument would be the same if the human race were reduced to a million or ten million.
Suppose that a blind person in a first world country wants help paying for a guide dog and/or wants guide dogs for other blind people in first world countries, but has heard of effective altruism. What honest arguments could the blind person use?
If I were designing an intelligence, I’m not sure how much control I would give it over its own brain. People are already able to damage themselves pretty badly, even with the crude tools they’ve got. I would experiment with intelligent species to see how they’d behave with more control over their brains. What would you do?
Sidenote: Birds show some possibilities of making brains more efficient per weight.
TED talk about neurons and brains. This is not a great TED talk, but it’s got somewhat about comparisons between brains in different species, in particular that neuron size and density varies between species. Comparisons of brain size tells you less than people assume.
Brains and competition aren’t just about sexual selection: Females (especially) compete for resources to feed and care for themselves and their children. In some species, males also compete for resources for their children. Reproductive selection isn’t just about mating selection. See Mother Nature by Sarah Hrdy. Interview about humans as cooperative breeders
Do we need to think about hardware, software, and firmware (at least) for brains, rather than just hardware and software?
[Sound cuts off at 38:00. comes back at 39:10]
How much of organisms consist of traits which aren’t being selected for?
The sound quality deteriorates enough at about an hour that I’m giving up.
This sounds like motivated cognition. “How can I use EA to justify what I already want to do?”
It seems that “donate to a guide dog charity” and “buy me a guide dog” are pretty different w/r/t the extent that it’s motivated cognition. EAs are still allowed to do expensive things for themselves, or even as for support in doing so.
On second thought, the blind person’s appeal might best be “buy your warm fuzzies here”.
IAWY, though expensive interventions can be cost-effective for people with high productivity and people in the first world are a lot more productive than people people in the third world. If Maurice Hilleman went blind mid-career, getting him a seeing eye dog would be amazingly effective.
I suggest that “people in the first world are a lot more productive than people in the third world” might be better expressed as “people can be a lot more productive in the first world than in the third world”. If Maurice Hilleman had been born in some out-of-the-way village in the poorest parts of Africa, he almost certainly wouldn’t have achieved any of what he actually did.
This rephrasing has the advantage of being less likely to lead to heated arguments about “human biodiversity” and the like.
(It’s not perfect. It is likely true that the affluent Western countries afford much better opportunities for people to become very productive, and that most of the adults in that hypothetical out-of-the-way African village would be less productive if transplanted to, say, the USA than if they had been born there.)
I think we agree in sentiment, but I don’t want to understate the effect. Almost everyone in rich countries is more productive than almost everyone in poor countries. A subsistence farmer living on less than $1 per day can move to the US and earn $54 per day mowing Maurice Hilleman’s lawn. When you are surrounded by productive people and are swimming in capital, it’s not hard to be more productive than the medium earthling.
ETA: Rereading my comment and your reply: you’re right. My comment switches between a fungible idea of people, who are more productive some places than other, and a specific person (Maurice Hilleman) who is more productive than other persons in his environment. I’ll think about rephrasing.
Yes, all agreed.
Effective altruism doesn’t necessarily imply selflessness in all your actions, but effectiveness in actions that are intended to be altruistic.
The argument remains the same even if the odds are very low, because of the high stakes involved.
Only laypeople make such assumptions, see my other comment.
What’s wrong with the TED talk? Like most, it can be summarized in 5 sentences, but I think it is worth summarizing, which makes it one of the best ever:
When people talk about brain size, they mainly talk about brain mass, because that’s easy to measure. Wouldn’t neuron count be better? Primate neuron count is proportional to brain mass, but rodent neuron count is sublinear. Energy expenditure is proportional to neuron count. Humans can only afford to spend so much energy on their brains because of cooking.
The talk fails to address why rodents have bigger neurons, rather than small neurons in small brains. And whether rodents are typical in this pattern. The well-known pattern is that brain mass grows like a power (2/3 or 3⁄4) of body mass, and that the ratio is a good measure of intelligence. I think the implication is that neurons counts grow at a lower power, but still grow. I think that there is data at the web site of the speaker, Suzana Herculano-Houzel.
Might be a better idea to skim one of her papers: “The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost”, Herculano-Houzel 2012
Thanks. I think my problem is that I’m not especially familiar with the point of view she’s arguing against. However, I think it’s interesting that the unsourced estimated of neurons (100 billion) came in so close to the right number (86 billion).
You should also account for how myelinated the neurons are, and how many of them are not myelinated at all, since this affects neuron size and energy expenditure a lot.
Without myelin, the conduction speed of a neuron is mainly determined by its diameter. Some animals don’t have myelin at all, and their neurons can be huge because of this reason. I’m not sure if this applies to rodents, but it would explain why their neurons grow in size as the brain grows, because conduction distances grow too, and therefore you need a higher conduction speed, and this can only be achieved by having a larger neuron. Even humans have some neurons that are not myelinated, and they vary from this to highly myelinated with anything in between and their conduction speeds vary in proportion.
Connection count should also be an important factor, and it can vary by many orders of magnitude. It also matters how many neurons are active at a time.
ETA: removed it. Nutrition is the mind-killer.
Sources:
Axonal conduction delays
Principles underlying mammalian neocortical scaling
There’s another thing to consider, which is that myelination requires special glial cells around axons to actually provide the myelin sheath, and these cells are relatively large themselves and use up energy. Glia (both for myelin and many many other functions) actually take up a large fraction of the space of the human brain and there are (by some estimates) 10x more glia than neurons in the human brain. The human brain especially has a lot of glia (90% of brain tissue) whereas for a mouse it’s only 65% of brain tissue.
At last, a reply to the interesting stuff! :)
IIRC myelinated neurons are both more energy efficient taking into account the glial cells and take less space than nonmyelinated neurons if we talk about similar conduction velocities. I’ll have to check this one just to be sure.
You mean the volume or the number of cells? This is certainly interesting and supports my hypothesis of why rodent neurons grow in size as their brains grow. What do you think?
Why are some brains less myelinated than others? Was their evolution just less lucky?
Possibly, but I’d caution against simplistic evolutionary arguments; evolution is rarely so simple. For instance, chimpanzees have higher axon myelination during development and adulthood than human brains do.
Perhaps they’re the lucky ones in that particular case. We really can’t assume our brains are the most efficient in all respects.
A raw food diet that a modern consumer with access to supermarkets and kitchen equipment might try would not resemble a raw diet in pre-agricultural times. Most raw food diets strongly recommend juicing, blending, mixing, etc. which are essentially pre-digestion. Furthermore, they’re also generally billed as “weight loss diets,” allowing sedentary people to lose weight without exercising—the calorie total might be in the 1000-1500 range.
Cooking greatly increases how many things you can eat, what’s safe to eat, and how many calories you can fit into your stomach. Cooking might decrease nutrient absorption by 5-10%,but when you can eat 50-100% more, it’s a fine trade.
I think you guys are just privileging a cool hypothesis a neuroscientist is making about gastroenterology. It’s pretty weird and frustrating how many objections this single point is getting, given that just like I already said, it’s very testable.
So is chewing and it works just fine.
Who’s doing the billing? Are these people fat to begin with? Where are you getting these numbers from and do they really have anything to do with the foodstuff being raw?
Most foods you can make safe by cooking do not comprise a great deal of what people eat. Some of them could be unsafe raw just because human GI tracts are used to cooked food by now. Also, most foods don’t significantly shrink when you cook them.
Again, where are you getting these numbers from?
Are you familiar with raw food diets? Could you point to one? In particular, where do the bulk of the calories come from?
I can’t say I’m familiar with them, although I’ve occasionally eaten very little cooked food. Bulk of the calories could come from fish, milk, eggs, nuts, fruit and vegetables. Fish and eggs carry a risk of food poisoning if you don’t know your source. So does red meat.
It’s possible that this wouldn’t be a significant problem if your GI-tract and immune system were used to these kinds of threats however. It’s also possible that bacteria causing food poisoning are more potent and more common than they used to be or than they are in nature.
Living on a raw food diet when you don’t have to do your own hunting and gathering might have a different energy balance.
Could be so, but that’s idle speculation and I’d still test the hypothesis before accepting it :)
Are there any pre-modern societies which live on raw food?
Requesting a data point: did your heuristics point to me being a proponent of raw foodism and trying to sneak in my ideology? I hadn’t even read about it before I made my comment.
I don’t encounter many people in my life who care about particular diets much and count myself amongst them. In fact, I know none irl. Do you find this unusual?
I didn’t assume you were a raw food proponent.
I don’t know a lot of people who do much with their diet. Offhand, I know two ex-vegetarians who still eat very little meat, one person who did CR for a few years, and one person who gave himself an eating disorder (I hope he’s over it) by following a claimed-to-be- healthy diet which was much too low calorie for him. Oh, and a woman who seems to be doing well on the same diet. And two people who are following some complicated diet which involves different types of food on a cycle. They’ve lost weight, but I’m dubious about their ability to maintain it. Rather more people than I thought before I started listing.
I know two people who are serious about supplements. One of them limits carbs drastically.
“Nutrition is the mind-killer.” Definitely true, and funny.
I’ve done a little survey about the effects of trying to lose weight, if you’d like to see some mind-killing and some non-mind-killed people. I’m trying to figure out how to deal with (other people’s) compulsion to give advice. Forbid it, or let it roar?
I’m not sure why there not being any would tell us anything, because cooking is common knowledge everywhere by now, and there are other reasons to cook food than nutrition.
That only suggests that peripheral neurons should grow with body size, not neurons in the brain. I suppose that it’s simpler to let central and peripheral neurons be the same size, but that doesn’t seem to me like a good reason.
Why? If the brain is bigger, so are the distances within it.
Do you have a source for the quantitative variation of myelination between vertebrates, and not just the binary question of some have it (vertebrates), some don’t?
Do you have a source for connection count varying between species?
Here’s evidence that axonal diameter grows as brains grow, and that axonal diameter is important for conduction speed.
If you read that article, you will realize that the degree of myelination in a species will depend on how fast conduction they need, since if conduction speed is not taken into account, nonmyelinated fibers are much cheaper to have.
I didn’t have a source at hand for variation of connection count, but found this.
“The neocortex undergoes a complex transformation from mouse to whale. Whereas synapse density remains the same, neuron density decreases as a function of gray matter volume to the power of around −1/3.”
This means that synapse count per neuron varies considerably between species.
A public version of the third link
Thanks.
Do they?
(the Wikipedia section refers to raw vegan diets, but I suppose that eating significant quantities of raw meat, fish or dairy poses a risk of infection by bacteria or worms)
Your gut flora would probably be completely different and more resistant to those things if you ate all your meat raw. Are other animals affected by food poisoning to any significant degree?
AFAIK, the immune system has some ability to adapt to different pathogens, but this doesn’t mean that eating significant quantities of raw meat wouldn’t pose a health risk, particularly for a long-lived species such as humans.
AFAIK, trichinellosis is endemic in wild carnivorous mammals.
It definitely poses a risk, but how high is it?
The question is, do they actually suffer from it, or are they carriers.
It seemed more repetitious and less focused than most.
I’m thinking “Increasing independence for blind people in first world countries increases their economic impact, potentially from a net negative to a net positive. It can be expected that some of those blind people will contribute to charities. Information to the blind is often sent through extremely narrow channels controlled by a few organizations, so it would be somewhat trivial to promote EA while paying for guidedogs and associated training, increasing the potential pool of EA donors.”
It has flaws (probably more efficient to uplift the Occupy types by paying off their college debt and simultaneously evangelizing EA; the returns would almost definitely be higher), but that’s the first response that comes to mind.
If the increased economical impact is really likely, the effective altruist could give a loan to the blind person in the first-world country to buy the guide dog, and then donate the returned money to an efficient charity.
Perhaps a one-line, how-to-read-this-post would be helpful at the top? Btw, good questions!
What difficulties do you think need to be addressed?
Oh, I just got confused when I started reading the second paragraph and it seemingly had no connection to the first. Was thinking something like, “This is a series questions and notes I took while watching Eliezer’s talk.” at the top might make the format clearer to other readers. Though perhaps I should have inferred that from the title :)
Done—it’s not as though I always pay close attention to the title.
Awesome!
This sounds like it has the same failure modes as boxing. E.g. an AI doesn’t need direct Write access to its source code if it can manipulate its caretakers into altering it. Like boxing, it slows things down and raises the threshold of intelligence required for world domination, but doesn’t actually solve the problem.
You’d better prevent it from building a successor that’s just like itself but with certain modifications. To prevent that you need at least to prevent it from having absolute introspective access...
well, an intelligence that cannot self-modify (for a broad enough understanding of “self-modify”) is significantly less likely to get superintelligent (though of course it’s possible that I end up designing a superintelligence first time out of the park).
that said, “can’t modify its own brain” != “can’t self-modify” in that broad sense… if I can’t modify my own brain but I can create a copy of myself whose brain I can modify, most of the same difficulties arise. (Unless I happen to believe, like many humans do, that a copy of myself at time T is importantly different from me at time T, in which case maybe those difficulties don’t arise.)
The argument remains the same even if the odds are very low, because of the high stakes involved.
The question is only nontrivial if you presuppose that to hear of effective altruism is to agree with its tenets. Without such a presupposition honestly has nothing to do with it.
Fair enough. I was being mealy-mouthed. I really meant “what if the blind person either agrees with effective altruism and/or is trying to appeal to people who do?”
I didn’t buy the alleged advantage of a noise free environment. We’ve known since von-Neumann’s paper titled:
...that you can use unreliable computing components to perform reliable computation—with whatever level of precision and reliability that you like.
Plus the costs of attaining global synchrony and determinism are large and massively limit the performance of modern CPU cores. Parallel systems are the only way to attain large computing capacities—and you can’t guarantee every component in a large parallel system will behave deterministically. So: most of the future is likely to lie with asynchronous systems and hardware indeterminism, rather contrary to Yudkowsky’s claims.