1) Someone presents you with a huge deck of cards (not standard playing cards
-- each card has a spot of a given color on it). Before even one card is
seen,
what is the probability that the first card dealt is red?
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
2) Assuming you assigned some finite probability P(red), now for the same
card
that you still haven't seen, what is the probability that it it blue?
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
3) Now, what is the probability that it is yellow? Black? Purple? Orange?
White? Fuchsia? etc.? Has your P(red) assessment changed? How many colors
can you name? Are you willing to assign them equal probabilities just
based on
ignorance?
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
4) Now, suppose you are told reliably that every card in the deck is either
red, blue, or green. Now what is your P(red)?
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
5) One more bit of information now: among the blue cards, there are light
blue and dark blue. Does this change P(red)?
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
This thought problem illustrates some serious problems with the idea of a
completely uninformed prior probability. In the continuous world it isn't any
better -- what would be an uninformed prior over the set of real numbers? My
best answer is that every prior must be informed (or perhaps misinformed) by
one's current view of the universe, which reflects one's past experience. It
is inherently subjective, but it represents the best one can do in a bad
situation. All colors aren't equally likely based on our past experience --
primary colors show up far more frequently, as do white and black; we are
conditioned by experience with standard playing cards in a way that biases us
toward 50% red and 50% black; we are conditioned by our languages, which
divide
up the color space in different ways.
A good classical way to calibrate subjective probability is to create a
reference scale such as a set of 100-card decks, such that deck n has n red
and
100-n black cards. Now, offer one of the calibrated ("frequentist") decks as
Option A and the new, uncalibrated event as Option B. Whichever deck the
decision maker chooses, a card is sampled at random and if it is red, he
wins a
(hypothetical) dollar; so if the deck offered in Option A is deck 42 and the
subject believes there is more than 42% chance of sampling a red card from the
Option B deck, he will select Option B.
If you offer a separate bet with each of the calibrated decks, any even
minimially rational subject will choose Option B for all n up to some value
n0,
and Option A for all n greater than n0. This sets the subject's "subjective"
prior P(red) at approximately n0 percent (feel free to use larger decks if you
want arbitrarily fine calibration, but you get the general idea). Now all
this
has taken place before any card is actually sampled, so it is all
"uninformative" in the Bayesian sense. (Note - unless you are Bill Gates, I
don't recommend this as a practical assessment approach for large Bayes nets.)
All of this leads me to the conclusion that the "I don't know" response is
simply not permissible in the probabilistic context (they always know, they
are
just reluctant to say because that commits them to possible loss or
embarrassment if they happen to be wrong). The closer they come to
indifference, the harder it will be to make a judgment, but that doesn't mean
it can't be done. And of course in the deterministic context, "I don't know"
would be a given, otherwise why bring in probability at all?
Jonathan Weiss
At 6/6/99 11:40 PM, David Poole wrote:
>Rolf Haenni wrote:
>> But knowing that P(Q)=0.5 is
>> clearly not the same as to know nothing about Q.
>
>This isn't so clear. Here is some reasoning I have used to convince
>myself that these are indeed the same:
>
>Suppose we have a button, and when we press the button, there is one of
>two outcomes, either a H or a T (just to make it look familiar). Let the
>event Q be the outcome of pressing the button.
>
>Before we press any buttons, I ask you what will be the first outcome.
>You, of course, say "I don't know".
>
>Suppose we press the button 4 times and observe a H,T,T,H. And then I
>ask you what will be the outcome of the 5th press. You then would also
>say "I don't know".
>
>Now suppose that we were to press the button 1,000,000 times and we
>observe that half of them resulted in H and half resulted in T (and we
>could not detect a pattern in the sequence). If I then ask you to say
>what the outcome of the 1,000,001st press will be, you would also say "I
>don't know".
>
>The first "I don't know" was the result of knowing nothing about Q.
>The last "I don't know" was the result of knowing the P(Q) is 0.5.
>
>But it seems to me that these "I don't knows" are not of different
>types: you have no idea whether the outcome will be H or the outcome
>will be T initially or for the 1,000,001st press. [Of course you have
>very different knowledge about the probability of Q, but no one would
>claim any differently.] Parsimony would suggest that we don't need to
>distinguish these "I don't know"s (i.e,., the prediction from ignorance
>and the prediction from knowing the probability is 0.5).
>
>For those people who would like to distinguish ignorance for the outcome
>of a binary variables and probability 0.5, I would like to know how many
>different meanings are there to "I don't know" (for a binary random
>variables)? (If there are a finite number (such as two) different
>meanings, when in the above sequence of making predictions and pressing
>the button do the meanings switch? Is the "I don't know" prediction for
>the 5th toss have the same meaning as the first "I don't know" or the
>same meaning as the "I don't know" for the prediction of the 1,000,001st
>press? (Or isn't 1,000,000 presses enough data to warrant drawing a
>conclusion about the probability?).
>
>I would really like to know. There has been lots of research based on
>the distinction between ignorance and probability, yet this is some
>reasoning I have used to convince myself that the Bayesians are right.
>
>Thanks,
>David
>
>--
>David Poole, Office: +1 (604) 822-6254
>Department of Computer Science, poole@cs.ubc.ca
>University of British Columbia,
<http://www.cs.ubc.ca/spider/poole>http://www.cs.ubc.ca/spider/poole
>