Rational thought needs a knowledge base; given that, it can outperform evolution. When the knowledge base is lacking and improving it is difficult, then an evolutionary strategy may be the best course. Lots of examples of genetic algorithms accomplishing what rational design couldn’t (with the current knowledge base) at TalkOrigins.
Cyan2
Karma: 365
David MacKay did a paper on this. Here’s a quote from the abstract:
If variation is produced by mutation alone, then the entire population gains up to roughly 1 bit per generation. If variation is created by recombination, the population can gain O(G^0.5) bits per generation.
G is the size of the genome in bits.
Rolf,
If you look at equation 3 of MacKay’s paper, you’ll see that he defines information in terms of frequency of an allele in a population, so you’d have to provide a whole population of randomized hard drives, and if you did so, the population would have zero information.
Rolf,
Would you agree that the information-theoretic increase in the amount of adaptive data in a single organism is still limited by O(1) bits in Mackay’s model?
I can’t really process this query until you relate the words you’ve used to the math MacKay uses, i.e., give me some equations. Also, Eliezer is pretty clearly talking about information in populations, not just single genomes. For example, he wrote, “This 1 bit per generation has to be divided up among all the genetic variants being selected on, for the whole population. It’s not 1 bit per organism per generation, it’s 1 bit per gene pool per generation.”
Eliezer,
I’ve thought hard about your reply, but it’s not clear to me what the distinction is between bits on a hard drive (or in a genome) and information-theoretic bits. One bit on a hard drive answers one yes-or-no question, just like an information-theoretic bit.
The third section of the paper is entitled “The maximum tolerable rate of mutation”. (MacKay left the note “This section needs checking over...” immediately under the title, so there’s room for doubt about his argument.) MacKay derives the rate of change of fitness in his models as a function of mutation rate. He concludes (as you did) that the maximum genome size scales as the inverse of the mutation rate, but only when mutation is the sole source of variation. He makes the claim that maximum genome size scales as the inverse of the square of the mutation rate when crossover is used.
It seems to me that this is a perfect example of your idea that one doesn’t really understand something until the equations are written down. MacKay has tried to do just that. Either his math is wrong, or the idea that truncation can only give on the order of one bit of selection pressure is just the wrong abstraction for the job.
(Just as a follow up, MacKay demonstrates that the key difference between mutation and crossover is that the full progeny (i.e., progeny before truncation) generated by mutation have a smaller average fitness than their parents, while the full progeny generated by crossover have average fitness equal to their parents’.)
“If you flip a base and it doesn’t make any difference, then you’ve just proved that it was junk-DNA, right?”
Not quite. Certain bases in the protein-coding sections of genes (i.e., definitely not junk DNA!) can be flipped without changing the resulting proteins. This can happen because there are 64 different codons, but only 20 different amino acids are used to build proteins, so the DNA code is not one-to-one.
It might be safer to say that if you delete the base and it makes no difference, then it was junk, but even this will run into problems...
logicnazi,
Wei Dai’s post doesn’t make sense except in the context of MacKay’s paper. If you’ve read the paper thoroughly, it should be pretty clear what he’s talking about.
The fact that the MacKay’s fitness function is the distance to a “master” genome has nothing to do with how much information god could convey to someone. It’s just a way to model constant environmental conditions, like the sort of thing that has kept modern sharks around since they evolved 100 million years ago.
His model about as simplified as one could get and still have descent with modification and selection, but it’s entirely adequate for the limited purpose for which I brought it up in this conversation. That is, it contains the minimal set of features that a system needs to be covered by Eliezer’s dictum of 1 bit gained per generation; therefore we can use it to test the assertion just by running the model. This is just what Wei Dai has done—nothing more and nothing less. In particular, the model has no notion of complexity, but it doesn’t need one for us to test Eliezer’s assertion.
michael vassar,
“To make the simulation really compelling it has to include some sort of assortative mating.”
Meh. Assortive mating can decrease or increase the variance of the progeny, depending on whether the sorting is by similarity or dissimilarity, respectively. I’m happy with random mating as a first step.
MacKay comments on Kimura’s and Worden’s work and its relation to his own on page 12 of the paper. In particular, he notes that in Worden’s model, fitness isn’t defined as a relative quality involving competition with other individuals in the population; rather, one’s genotype determines the probability of having children absolutely. MacKay says that this is how Worden proves a speed limit of one bit per generation even with sexual reproduction, but he doesn’t do any math on the point.
″...mathematics that represent continuous scales which would be best represented by the real numbers system with the limited significant digits.”
If you limit the number of significant digits, your mathematics are discrete, not continuous. I’m guessing the concept you’re really after is the idea of computable numbers. The set of computable numbers is a dense countable subset of the reals.
Benoit Essiambre,
Right now Wikipedia’s article is claiming that calculus cannot be done with computable numbers, but a Google search turned up a paper from 1968 which claims that differentiation and integration can be performed on functions in the field of computable numbers. I’ll go and fix Wikipedia, I suppose.
Benoit,
In the decimal numeral system, every number with a terminating decimal representation also has a non-terminating one that ends with recurring nines. Hence, 1.999… = 2, 0.74999… = 0.75, 0.986232999… = 0.986233, etc. This isn’t a paradox, and it has nothing to do with the precision with which we measure actual real things. This sort of recurring representation happens in any positional numeral system.
You seem very confused as to the distinction between what numbers are and how we can represent them. All I can say is, these matters have been well thought out, and you’d profit by reading as much as you can on the subject and by trying to avoid getting too caught up in your preconceptions.
rukidding,
The immigration you refer to took place 10,000 years ago, and (this is key!) the land was uninhabited by humans when those Asian immigrants took up residence. But whether you call them natives or Asian immigrants, European settlers slaughtered them and claimed their lands.
Jared Diamond, in his book The Third Chimpanzee, quotes several famous Americans on the subject of the native peoples. Here are three quotes:
George Washington: “The immediate objectives are the total destruction and devastations of the settlements. It will be essential to ruin their crops in the ground and prevent their planting more.”
Andrew Jackson: “They have neither the intelligence, the industrym the moral habits, not the desire of improvement which are essential to any favorable change in their condition. Established in the midst of another and a superior race, and without appreciating the causes of their inferiority or seeking to control them, they must necessarily yield to the force of circumstances and ere long disappear.”
Theodore Roosevelt: “The settler and pioneer have at bottom had justice on their side; this great continent could not have been kept as nothing but a game preserve for squalid savages.”
The other quotes, from Benjamin Franklin, Thomas Jefferson, John Quincy Adams, James Monroe, John Marshall, William Henry Harrison, and Philip Sheridan, are just as damning.
Erratum: George Washington said, ”...their settlements...”, not, ”...the settlements...”
Constant, a reply in brief:
“unkind words literally kill people dead”
Incitements to violence by leading citizens may plausibly be inferred to cause death. This is not usually classed as a bullshit inference.
“the unkind words you quote were… cherry-picked”
You say cherry-picked, I say representative of government policies that were actually carried out. Tomato, tomahto.
“native americans were on their side entirely without sin… never gave whites any reason to think of them as enemies.”
As Nick Tarleton noted, I never made that claim.
By “damning”, I meant, “worthy of condemnation as harmful, illegal, or immoral.” (That’s pretty much straight from the dictionary.)
Let me just add, genocide is something humans do—everywhere, at all times in history. (Chimps too, less efficiently.) The natives were no better or worse than the settlers, only more poorly equipped.
I ran across a curious misunderstanding of probability in the SF novel Diamond Mask. In the murder mystery plotline of the book, the protagonist had collected and analyzed data on an (implicitly mutually exclusive and exhaustive) list of eight or nine suspects. The author used probabilities of lower than 20% as a shorthand for not too likely, probabilities of between 20% and 50% as moderately likely, and probabilities above 50% as indicating prime suspects. Unfortunately, there was ~300% total probability in the list. The author could have gotten away with it if she’d just used the word “likelihood” instead of “probability”.
“Affective death spiral” sounds like the process by which I became a militant evangelical Bayesian. But I got better: now I’m only a fundamentalist Bayesian, and my faith does not require me to witness the Bayesian Gospel to those who aren’t interested.
At best, out of N formulas, each has a 1/N chance of being correct. (At worst, none of the formulas is correct.)
Technical note: Occam factors (and prior probabilities generally) can cause these chances to deviate from 1/N.
“They usually resort to the script of presuming a personal insult” instead of rightly apprehending the point you’re making, which is...?
This is the difficulty I have with your comments, Caledonian. You always leave the interesting part out. (This is not a personal insult, by the way—just a straightforward observation.)
Q, Eliezer’s probabilities are Bayesian probabilities. (Note the “Bayesian” tag on the post.)
Cumulant-nimbus,
There’s no shortage of statisticians who would disagree with your assertion that the probability of a probability is superfluous. A good place to start is with de Finetti’s theorem.
Constant,
I can’t speak for other Bayesians, but I prefer to use the idea of Bayesian probabilities encode “states of information” as opposed to “degrees of belief”. To me, overcoming bias means making sure your beliefs reflect the actual information available to you, so I prefer to use a phrase which directs attention to that information immediately. This isn’t my idea; it’s one of the key ideas put forward by E. T. Jaynes. To get a sense of how this works in a geometric problem similar to the Buffon needle problem, I recommend Jaynes’s paper The Well-Posed Problem.