Haipeng,
The question of evaluating the quality of an approximation to
probabilistic inference is really a very important one and a difficult
one, with which we also struggle with in Irvine.
Sometimes it is possible to generate by an approximation an upper and
lower bounds of the exact probability which can bound provide some
estimate of the accuracy.
However, such methods are often very inaccurate.
Therefore one produces arbitrary approximations (no guarantees).
In that case, I dont see any other way but to test your approximation
algorithm on relatively small or moderate networks and compare against
the exact figure (computed by an exact algorithm) with the
hope that the information gained from such comparison scales up.
You can then try too show superiority relative to other
competing approximation algorithms.
For some recent relevant work in this area please see:
http://www.ics.uci.edu/~dechter/publications/
R95.
I have seen some people comparing approximation reults to each other
(comparing to Gibbs sampling for instance) however such comparisons are
meaningless, I think.
----Rina.
Haipeng Guo wrote:
>
> Hi, there,
>
> To measure the accuracy of an approximate Bayesian inference algorithm,
> many people use either Mean Square Error(MSE) or cross-entropy between the
> exact and the approximate marginal probabilities of all nodes. But there
> is a possibility that for some very large networks the exact marginals
> can't be computed at all. My question is how to evaluate the algorithm's
> perfomance for these very large networks for which eaxct solution is not
> available at all?
> Thank you very much if you would provide any references to this question.
>
> cheers,
> -hpguo
>
> KDD Lab, CIS Dept., Kansas State University
This archive was generated by hypermail 2b29 : Mon May 07 2001 - 12:02:39 PDT