[John Wilkins (1614-1672)] set out to determine how a restricted set of symbols — perhaps just two, three, or five — might be made to stand for a whole alphabet. They would have to be used in combination. For example, a set of five symbols — a, b, c, d, e — used in pairs could replace an alphabet of twenty-five letters...
...So even a small symbol set could be arranged to express any message at all. However, with a small symbol set, a given message requires a longer string of characters — “more Labour and Time,” he wrote. Wilkins did not explain that 25 = 52, nor that three symbols taken in threes (aaa, aab, aac,…) produce twenty-seven possibilities because 33 = 27. But he clearly understood the underlying mathematics. His last example was a binary code, awkward though this was to express in words:
Two symbols. In groups of five. “Yield thirty two Differences.”
That word, differences, must have struck Wilkins’s readers (few though they were) as an odd choice. But it was deliberate and pregnant with meaning. Wilkins was reaching for a conception of information in its purest, most general form. Writing was only a special case: “For in the general we must note, That whatever is capable of a competent Difference, perceptible to any Sense, may be a sufficient Means whereby to express the Cogitations.” A difference could be “two Bells of different Notes”; or “any Object of Sight, whether Flame, Smoak, &c.”; or trumpets, cannons, or drums. Any difference meant a binary choice. Any binary choice began the expressing of cogitations. Here, in this arcane and anonymous treatise of 1641, the essential idea of information theory poked to the surface of human thought, saw its shadow, and disappeared again for four hundred years.
The global expansion of the telegraph continued to surprise even its backers. When the first telegraph office opened in New York City on Wall Street, its biggest problem was the Hudson River. The Morse system ran a line sixty miles up the eastern side until it reached a point narrow enough to stretch a wire across. Within a few years, though, an insulated cable was laid under the harbor. Across the English Channel, a submarine cable twenty-five miles long made the connection between Dover and Calais in 1851. Soon after, a knowledgeable authority warned: “All idea of connecting Europe with America, by lines extending directly across the Atlantic, is utterly impracticable and absurd.” That was in 1852; the impossible was accomplished by 1858, at which point Queen Victoria and President Buchanan exchanged pleasantries and The New York Times announced “a result so practical, yet so inconceivable … so full of hopeful prognostics for the future of mankind … one of the grand way-marks in the onward and upward march of the human intellect.” What was the essence of the achievement? “The transmission of thought, the vital impulse of matter.” The excitement was global but the effects were local. Fire brigades and police stations linked their communications. Proud shopkeepers advertised their ability to take telegraph orders.
Information that just two years earlier had taken days to arrive at its destination could now be there—anywhere—in seconds. This was not a doubling or tripling of transmission speed; it was a leap of many orders of magnitude. It was like the bursting of a dam whose presence had not even been known.
And:
Szilárd — who did not yet use the word information — found that, if he accounted exactly for each measurement and memory, then the conversion could be computed exactly. So he computed it. He calculated that each unit of information brings a corresponding increase in entropy—specifically, by k log 2 units. Every time the demon makes a choice between one particle and another, it costs one bit of information. The payback comes at the end of the cycle, when it has to clear its memory (Szilárd did not specify this last detail in words, but in mathematics). Accounting for this properly is the only way to eliminate the paradox of perpetual motion, to bring the universe back into harmony, to “restore concordance with the Second Law.”
Szilárd had thus closed a loop leading to Shannon’s conception of entropy as information. For his part, Shannon did not read German and did not follow Zeitschrift für Physik. “I think actually Szilárd was thinking of this,” he said much later, “and he talked to von Neumann about it, and von Neumann may have talked to Wiener about it. But none of these people actually talked to me about it.” Shannon reinvented the mathematics of entropy nonetheless.
And:
Solomonoff, Kolmogorov, and Chaitin tackled three different problems and came up with the same answer. Solomonoff was interested in inductive inference: given a sequence of observations, how can one make the best predictions about what will come next? Kolmogorov was looking for a mathematical definition of randomness: what does it mean to say that one sequence is more random than another, when they have the same probability of emerging from a series of coin flips? And Chaitin was trying to find a deep path into Gödel incompleteness by way of Turing and Shannon—as he said later, “putting Shannon’s information theory and Turing’s computability theory into a cocktail shaker and shaking vigorously.” They all arrived at minimal program size. And they all ended up talking about complexity.
And, an amusing quote:
The key to [quantum] teleportation and to so much of the quantum information science that followed is the phenomenon known as entanglement. Entanglement takes the superposition principle and extends it across space, to a pair of qubits far apart from each other. They have a definite state as a pair even while neither has a measurable state on its own. Before entanglement could be discovered, it had to be invented, in this case by Einstein. Then it had to be named, not by Einstein but by Schrödinger. Einstein invented it for a thought experiment designed to illuminate what he considered flaws in quantum mechanics as it stood in 1935. He publicized it in a famous paper with Boris Podolsky and Nathan Rosen titled “Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?” It was famous in part for provoking Wolfgang Pauli to write to Werner Heisenberg, “Einstein has once again expressed himself publicly on quantum mechanics… As is well known, this is a catastrophe every time it happens.”
From Gleick’s The Information:
More (#1) from The Information:
And:
And:
And, an amusing quote: