Folks,
For a bunch of decision theorists, we sure do fall easily into
alternative-focused thinking!!!
Let's try some value-focused thinking instead. What are the values
we are attempting to serve with the structure of our review process?
Values I have heard people articulate are:
- Quality. We want the set of accepted papers to be of uniformly high quality.
- Fairness. We want authors to have a fair chance of having their
work accepted on its merits whether or not they are well known to the
insiders in the UAI community.
- Openness to new ideas. We want outsiders with interesting new
ideas to have a fair hearing and a chance to have their papers
accepted.
- Cost in time and effort. We do not want the process to be overly
burdensome on authors, reviewers, and the program chairs.
- Cost in dollars. Obviously, cost in time and effort to volunteer
reviewers and chairs can be reduced by paying professionals to manage
certain aspects of the conference. This cost would have to be borne
either by conference registrants, by our corporate sponsors, or by
charging dues to join AUAI.
Can anyone add to the list?
We might consider creating a matrix with major values as the rows and
options as the columns. One option obviously is no change to current
processes. Other options would involve different modifications aimed
at improving our score on some of the rows, but possibly at the
expense of the score on other rows. The proponent of a given
modification might describe his/her modified process in enough detail
to provide an assessment of how it would score on each of the rows.
In particular, can proponents of double-blind reviewing describe
processes that we could implement at relatively low cost and would
provide significant improvement on some of the key attributes of
value?
Then we could focus our discussion on what our relative weights are
as a community on the different attributes of value, and on our
assessments of how well each of the options under consideration
scores on the different attributes of value.
Once we have structured the problem, we might want to consider taking
a survey to assess the community's views on the subject. Putting a
survey form up on the web is a relatively easy matter. Of course we
wouldn't have a scientific sampling process, but it would still be a
more reliable indicator of community views than the opinions of those
of us who tend to pipe up on email listservs. :-)
Kathy
This archive was generated by hypermail 2b29 : Mon Nov 19 2001 - 07:15:31 PST