I couldn't help participating in the discussion on this one. Aren't
we mixing our intuitive understanding of "I do not know" with the
fundamental consequences of laws of nature here? The problem in the
above example is that we *do know* that the coin will be tossed
1,000,000 times. Even this scant information is enough not to bet
large amount of money on the event "there will be between 450,000 and
550,000 in the next million coin tosses".
To put it in other words, according to the central limit theorem, any
slight bias in the probability of the coin toss will be magnified
significantly over the 1,000,000 tosses. This is already enough not
to bet large sums of money.
In fact, the central limit theorem is a strong counterexample to the
above Joe's thesis about "I do not know". To put it in simplified
form, it says that if we know *only* the mean and deviation of a
single experiment distribution (it does not have to be normal!!!), the
cumulative outcome of 1,000,000 repeated experiments will be a normal
distribution (isn't it amazing???). Thus, without any advance
information, just based on the fundamental laws of statistics, we can
infer some new properties of the probability distribution function of
1,000,000 tosses!
I am a physicist by most of my education (besides my Ph.D. with Daphne
Koller) and a strong advocate of Jaynes. The above type of reasoning
was prevalent in the theoretical physics of the going away XX-th
century (maybe because this is the *only* way to infer information
from "I do not know"). An example of this type of reasoning would be
the approach to non-relativistic theoretical mechanics by Landau and
Lifshits, where they infer the Newton laws just from translational
symmetry of space and time:
@Book{boo:LL:quant,
author = "Landau, Lev Davidovich",
title = "Quantum mechanics : non-relativistic theory",
publisher = "Pergamon Press",
year = 1977,
address = "Oxford, New York"
}
I think this type of inference is one of the major achievements of the
XX-th century science and this is what Jaynes meant to say.
-- Alexander V. Kozlov | alexvk@engr.sgi.com | (650) 933-8493