Re: "information gained" sometimes != "entropy reduction" ??
Ross D. Shachter (shachter@stanford.edu)
Tue, 11 Aug 1998 23:52:03 -07
Robert,
Let me put one more spin on the answers you have received so far. Let us
distinguish between the expected utility or entropy "ex ante," before you
make the observation, and "ex post," after you have made that observation.
The expected entropy ex ante cannot increase, but it might not be true for
every observation ex post.
In Stephen's example, if you observe X=1 then the entropy increases from
.81 to 1 bits; otherwise it decreases to 0 bits. Since X is equally likely
to be 0 or 1, the expected entropy decreases from .81 to .5 bits ex ante.
In Eric's utility statement, you are never worse off with an additional
observation ex ante, although your expected utility can decrease given a
specific observation.
Ross