Confidence

[Mr Fortuner connection modem]

Looking at the very interesting data sent by Eric Holman ("fewer than half
(24/65) of the vouchers from the same specimens were independently classified
in the same species"), two questions come to mind:
1) What would have been the
confidence factors attributed by the identifiers to the 24 species they
classified in the same species, and to the other 41 other species? Obviously,
it's too late to compute such factors, but I bet they would have been higher
in the first group of species.
2) Why were some species misidentified?
Identifiers not expert enough? Specimens in poor condition? Use of difficult
characters? In other words, look at the confidence factors and try to find out
what went wrong. I am afraid I don't agree with Michael Hubbard: differences
of opinion among experts do not make confidence scales meaningless; on the
contrary, confidence factors should be used to try to understand such
differences of opinion.

Now, all this discussion on confidence makes me
realize that there are three levels in confidence factors. The first level of
confidence corresponds to what I called endorsement factors in an earlier
message, that is, to factors attached to the characters that were used during
the identification.

A second level of confidence should then be considered,
corresponding to the identification session as a whole. The first factor in
this second level is: what identification method was used? Confidence in an
identification made using only a high risk method (in particular, any
elimination method) should be lower than if other, less risky methods are used
or, better, if a combination of several methods was used (provided they all
arrive at the same answer!). The second factor is the number of characters
used. Generally speaking, the more characters used, the better, but it's also
a question of using the right characters (those that are used in the specific
diagnoses of all the species in a genus). A third factor could be the use (or
not) of reference specimens as proposed by  Mike and Donna Ivie and others
(but I have some reservations on that point).

The third and last (?) level
of confidence would be attached to the number of persons who agreed with a
particular identification. When a dangerous pest is identified at the
diagnostic labs of California Department of Food and Agriculture, the
identification must be checked and approved by another scientist before any
regulatory action is taken. A description database should include fields in
which could be entered the successive identifications of the same specimens as
made by different identifiers (name of identifier, name of species, confidence
factor). If an agency really considers a particular identification to be a
matter of life and death, it should send the specimens to several identifiers
and see if they agree. If they don't, the reasons for their disagreement
should be examined as discussed above, by analyzing what went wrong at
confidence levels one and two.

Renaud Fortuner
fortuner at math.u-bordeaux.fr