Re: "information gained" sometimes != "entropy reduction" ??

Finn Verner Jensen (fvj@cs.auc.dk)
Tue, 25 Aug 1998 12:48:34 +0200

I have not followed the discussion from the start, but it seems that there is
a confusion between the expected entropy when gaining information and the
actual entropy when you have received the information.

If you have a possible source of information and you take the entropy for all
possible answers and average with the probabilities of these answers, then
this mean is never larger than your present entropy.

Certainly, the entropy may increase with specific answers.

I appologize, if my comment is out of focus.

/Finn V. Jensen