ksvanhorn

Karma: 509

ksvanhorn 10 Jan 2011 19:26 UTC
5 points
on: Occam’s Razor
“Different inductive formalisms are penalized by a worst-case constant factor relative to each other”

You mean a constant term; it’s additive, not multiplicative.

ksvanhorn 13 Jan 2011 16:30 UTC
3 points
in reply to: tcpkac’s comment on: Mutual Information, and Density in Thingspace
“The important caveat is : ‘boundaries around where concentrations of unusually high probability density lie, to the best of our knowledge and belief’ .”

I would call the above an instance of the Mind Projection Fallacy, as you seem to be assuming a probability density that is a property of the physical world, and which we are trying to ascertain. But probabilities are properties of our minds (or ideal, perfectly rational minds), not of the exterior world, and a probability distribution is simply an entity to describe our state of information; it is “the best of our knowledge and belief”.

ksvanhorn 13 Jan 2011 19:39 UTC
3 points
on: Beautiful Probability
“If anyone should ever succeed in deriving a real contradiction from Bayesian probability theory [...] then the whole edifice goes up in smoke. Along with set theory, ’cause I’m pretty sure ZF provides a model for probability theory.”

If you think of probability theory as a form of logic, as Jaynes advocates, then the laws and theorems of probability theory are the proof theory for this logic, and measure theory is the logic’s model theory, with measure-theoretic probability spaces (which can be defined entirely with ZF, as you suggest) being the models.

ksvanhorn 18 Jan 2011 2:45 UTC
−2 points
on: What Evidence Filtered Evidence?
“Has it ceased to be true that, in such-and-such a proportion of Everett branches or Tegmark duplicates in which box B has a blue stamp, box B contains a diamond?”

I am baffled as to why a person who calls himself a Bayesian continually resorts to such frequentist thinking. And in this case, it’s spectacularly inappropriate frequentist reasoning. Unless someone used a Geiger counter to decide whether or not to put a diamond in the box, quantum-level uncertainty is utterly irrelevant to this problem.

ksvanhorn 18 Jan 2011 2:55 UTC
4 points
in reply to: MoreOn’s comment on: Rationalization
What altitude were you at?

ksvanhorn 18 Jan 2011 4:42 UTC
5 points
in reply to: Eliezer Yudkowsky’s comment on: Avoiding Your Belief’s Real Weak Points
??? Jaynes died in 1998. It strains credulity to imagine that he wouldn’t have been aware of the MWI.

ksvanhorn 18 Jan 2011 19:35 UTC
40 points
0
in reply to: poke’s comment on: Is That Your True Rejection?
“There’s two things you get from a formal education: one is broad, you’re exposed to a variety of subject matter that you’re unlikely to encounter as an autodidact;”

As someone who has a Ph.D., I have to disagree here. Most of my own breadth of knowledge has come from pursuing topics on my own initiative outside of the classroom, simply because they interested me or because they seemed likely to help me solve some problem I was working on. In fact, as a grad student, most of the things I needed to learn weren’t being taught in any of the classes available to me.

The choice isn’t between being an autodidact or getting a Ph.D.; I don’t think you can really earn the latter unless you have the skills of the former.

ksvanhorn 20 Jan 2011 4:54 UTC
14 points
in reply to: kurige’s comment on: Moore’s Paradox
“I chose to believe in the existance of God—deliberately and conciously.”

I cannot conceive of how it is possible to deliberately and consciously choose to believe in something.

I grew up in a religious family. I served as a missionary for my church. I married a woman of the same religion. For most of my first 28 years I believed not only that there was a God but that he had established the church of which I and my family were all members.

But once I started examining my beliefs more closely, I realized that I was engaging in the most dishonest sort of special pleading in their favor. And then it was no longer possible to continue believing.

ksvanhorn 21 Jan 2011 4:39 UTC
4 points
on: 0 And 1 Are Not Probabilities
For any state of information X, we have P(A or not A | X) = 1 and P(A and not A | X) = 0. We have to have 0 and 1 as probabilities for probability theory even to work. I think you’re taking a reasonable idea—that P(A | X) should be neither 0 nor 1 when A is a statement about the concrete physical world—and trying to apply it beyond its applicable domain.

ksvanhorn 21 Jan 2011 19:41 UTC
3 points
on: The Fallacy of Gray
Thanks, Eliezer, for an excellent article. Some of my favorite quotables:
- the Quantitative Way
- Everything is shades of gray, but there are shades of gray so light as to be very nearly white, and shades of gray so dark as to be very nearly black.
- If science is a religion, it is the religion that heals the sick and reveals the secrets of the stars.
- “Everyone is imperfect” is an excellent example of replacing a two-color view with a one-color view.

ksvanhorn 21 Jan 2011 20:18 UTC
2 points
on: Something to Protect
The proper choice between (1) certainly save 400 lives and (2) 90% probability of saving 500 lives with 10% probability of saving no lives, depends on your utility function, which depends on the circumstances. If your utility is proportional to the number of lives saved, then sure, go with (2).

On the other hand, suppose that some cataclysm has occurred, those 500 lives are all that remains of the human race, and extinction of the human race has such an extremely negative utility for you that all other considerations amount to rounding error in the utility function. Then, to a close approximation, you want the choice C that maximizes P(S | C), where S=”human race survives”.

We have

P(S | C=1) = P(S | N=400)

P(S | C=2) = 0.9 * P(S | N=500)

where N is the size of the current population. Therefore, you should choose (1) if

P(S | N=400) / P(S | N=500) > 0.9.

ksvanhorn 22 Jan 2011 6:06 UTC
24 points
0
in reply to: GBM’s comment on: Probability is in the Mind
“Is it not true that when one rolls a fair 1d6, there is an actual ¹⁄₆ probability of getting any one specific value?”

No. The unpredictability of a die roll or coin flip is not due to any inherent physical property of the objects; it is simply due to lack of information. Even with quantum uncertainty, you could predict the result of a coin flip or die roll with high accuracy if you had precise enough measurements of the initial conditions.

Let’s look at the simpler case of the coin flip. As Jaynes explains it, consider the phase space for the coin’s motion at the moment it leaves your fingers. Some points in that phase space will result in the coin landing heads up; color these points black. Other points in the phase space will result in the coin landing tails up; color these points white. If you examined the phase space under a microscope (metaphorically speaking) you would see an intricate pattern of black and white, with even a small movement in the phase space crossing many boundaries between a black region and a white region.

If you knew the initial conditions precisely enough, you would know whether the coin was in a white or black region of phase space, and you would then have a probability of either 1 or 0 for it coming up heads.

It’s more typical that we don’t have such precise measurements, and so we can only pin down the coin’s location in phase space to a region that contains many, many black subregions and many, many white subregions… effectively it’s just gray, and the shade of gray is your probability for heads given your measurement of the initial conditions.

So you see that the answer to “what is the probability of the coin landing heads up” depends on what information you have available.

Of course, in practice you typically don’t even have the lesser level of information assumed above—you don’t know enough about the coin, even in principle, to compute which points in phase space are black and which are white, or what proportion of the points are black versus white in the region corresponding to what you know about the initial conditions. Here’s where symmetry arguments then give you P(heads) = ¹⁄₂.

ksvanhorn 22 Jan 2011 6:21 UTC
3 points
in reply to: Constant2’s comment on: Probability is in the Mind
“But frequentists emphatically are not talking about individual tosses. They are talking about infinitely repeated tosses.”

These infinite sequences never exist, and very often they don’t even exist approximately. We only observe finite numbers of events. I think this is one of the things Jaynes had in mind when he talked about the proper handling of infinities—you should start by analyzing the finite case, and look for a well-defined limit as n increases without bound. Unfortunately, frequentist statistics starts with the limit at infinity.

As an example of how these limiting frequencies taken over infinite sequences often make no sense in real-world situations, consider statistical models of human language, such as are used in automatic speech recognition. Such models assign a prior probability to each possible utterance a person could make. What does it mean, from a frequentist standpoint, to say that there is a probability of 1e-100 that a person will say “The tomatoe flew dollars down the pipe”? There haven’t been 1e100 separate utterances by all human beings in all of human history, so how could a probability of 1e-100 possibly correspond to some sort of long-run frequency?

ksvanhorn 27 Jan 2011 1:58 UTC
0 points
in reply to: Aurini’s comment on: Personal decisions the leading cause of death
That’s a pretty negative assessment. I see it more good news and useful information: “Pay attention to these things and you (probably) get to live longer.”

ksvanhorn 6 Feb 2011 6:55 UTC
−1 points
on: Distinct Configurations
Eliezer writes:

“It’s as if mere possibilities could have causal effects, in defiance of what the word “real” is usually thought to mean...”

Actually, mere possibilities can make a difference… if you have effects that propagate backwards in time. Here’s why.

To simplify, assume that you have a physical system that takes inputs (w,x) and produces outputs

(y,z) = F(w,x),

where w is an initial condition, x = z is an effect propagated back in time, and y is that portion of the system’s output that is not propagated back in time. Then given a specific value of w as an initial condition, whatever happens is a solution to the fixed-point equation

(y,x) = F(w,x)

for x.

Given w, a “mere possibility” is a possible output of F that could occur for the right choice of x, but doesn’t occur for the actual value of x. But these “mere possibilities” are properties of F. Changing the set of “mere possibilities” means changing the function F, and possibly getting a different fixed-point.

Why is this relevant to QM? Well, Cramer’s Transactional Interpretation of QM uses both the retarded and advanced wave solutions to Schrodinger’s equation. That is, it has “offer” waves going forward in time and “confirmation” waves going backwards in time. And I’m told that Aharonov’s work in QM also postulates subtle influences propagating backwards in time.

ksvanhorn 6 Feb 2011 7:33 UTC
5 points
1
on: Classical Configuration Spaces
Conditional independence does not require a rectangular “cloud of uncertainty”. If you have independent normal distributions for A and B, the joint distribution—your “cloud of uncertainty”—is an axis-parallel ellipse.

ksvanhorn 7 Feb 2011 8:19 UTC
0 points
on: The Born Probabilities
The Transactional Interpretation of QM resolves the mystery of where this nonlinear squared modulus comes from quite neatly. On that basis alone, I’m surprised that Eliezer doesn’t even mention it as a serious rival to MWI.

See http://www.npl.washington.edu/npl/int_rep/tiqm/TI_toc.html

ksvanhorn 7 Feb 2011 8:39 UTC
0 points
in reply to: endoself’s comment on: The Born Probabilities
Cramer argues that both Copenhagen and MWI are inconsistent with the results of the Afshar experiment.

ksvanhorn 23 Feb 2011 1:31 UTC
4 points
on: Timeless Beauty

So they assert that “the apple token is not meaningful by itself”, and then go on to say, “The meaning of the apple token emerges from its network of connections to other tokens.” This is not true reductionism. It is wrapping up your confusion in a gift-box.

Your criticism seems off the mark to me. These AI researchers were trying to automate reasoning. They turned to formal logic, which makes perfect sense—formal logic is just highly disciplined reasoning, so disciplined that a computer can check it. If you’re using formal logic, of course your symbols don’t mean anything of themselves; all the meaning is in the axioms, a.k.a. the symbols’ network of connections to each other.

A better criticism would be that they stuck to deductive logic—which only tells you when you can know something with absolute certainty—and took decades to realize that most of the useful real-world knowledge was probabilistic. One can’t formalize the concept of an apple because it is not well-defined, anymore than a blegg is.

ksvanhorn 24 Feb 2011 15:28 UTC
4 points
in reply to: JulianMorrison’s comment on: Science Doesn’t Trust Your Rationality

is it actually possible to use pure Bayesianism in any non-contrived, non-trivial context?

Sure. The difficulty of actually doing Solomonoff induction is irrelevant, because SI isn’t actually part of Bayesianism as everyone in the world except Eliezer Yudowsky defines it. Cox’s Theorem gives us the basic laws of probability, but there is nothing comparable telling us that algorithmic probability is the correct prior we should be using. A prior is an encoding of one’s prior knowledge / state of information before seeing the experimental data, and we have no a priori reason to expect simple explanations for everything.