Who is the postivist?

[Ted Schultz]

>Richard Zander wrote:
>Not as such. One might view, however, subclades that are identical in
>all trees with posterior probabilities that add to .5 or to, say, .95,
>as the equivalent (the first for more evidence for than against, the
>second to maybe actually call it a good estimation). Perhaps this would
>not work, what do you think?

I think you could do this in the same way as has been suggested for
bootstrap confidence tree "clouds" (see below).  However, I don't think it
would be fair to calculate this by giving equal prior probability to all
possible trees.

>Zander:
>> Any number of ways have been
>> proposed for calculating "confidence limits" around subtopologies have been
>> suggested (e.g., the Kishino-Hasegawa parametric test, Page's median trees,
>> parametric and non-parametric bootstraps).
>
>Interesting...can you supply a short bibliography? This seems to be a
>critical area we should all know more about.

OK, I'll try.  I hope no one will think I'm necessarily advocating any of
these methods, but I cite them as examples of how people are struggling
with the problem of assessing the posterior probabilities of various trees
and subtrees.

As far as branch support goes, everybody knows about decay indices (Bremer
supports) and bootstraps.  Both have problems, of course: How big of a
decay value is "good enough"?  What does the bootstrap mean?  (In a
separate thread in this group the point has been made that the bootstrap
does not tell us anything about how close to the "truth" our results are.)
If we think that in some way these measures are correlated with the
posterior probability of a monophyletic group, then trees in which branches
are collapsed when they fall below a certain critical value are actually
conglomerates of many multiple trees.  All of these possible trees together
form a "confidence set" such as you seem to be recommending.

Sanderson (1989.  Cladistics 5: 113) suggested that the parsimony tree be
regarded as a central tree around which there is an "error cloud"
(confidence set) of trees that includes all trees containing groups
occurring in 95% of the bootstrap replicates.  Page (1996. Cladistics 12:
83) suggested that the central tree should be the majority-rule bootstrap
consensus, and that some tree-distance metric (e.g., the partition metric)
be used to construct the error cloud around the central tree.  (This entire
scheme is applicable to ML as well as to parsimony trees.)

These approaches are unlikely to satisfy you, however, because the 95% set
of bootstrap trees is still a small subset of all possible trees, and you
seem to want to calculate probability as a function of all possible trees.

Confidence levels on decay indices have also been suggested by Faith
(Faith, D.P.  1991.  Cladistic permutation tests for monophyly and
nonmonophyly.  Systematic Zoology 40: 366-375.).  In this scheme, a null
distribution of possible length differences for a given branch is generated
based on a null of random character correlation.  Number and frequency of
character states is preserved within characters, but characters are
randomly assigned to taxa; many (100 or more) such permutations are used to
generate the null distribution, and the position of the observed length
difference (decay index) is found in this null distribution.  If it is
found to be significantly longer than our chance expectation, it is deemed
to have significantly high support.  A number of criticisms have been
leveled against the premise of this test (e.g., Carpenter in Cladistics,
Swofford et al. in Systematic Biology).  One practical reason is that it is
simply too easy for a decay index value to pass the test (i.e., the null of
"this is a sufficiently large support value" is subject to Type 2 error),
but this can be remedied by making it a two-tailed rather than a one-tailed
test.

Again, aside from its other problems this test will not satisfy you if you
want to afford any significant prior probability to trees beyond those that
could possibly be generated by any permutation of the particular data
matrix being considered.

There are a variety of other tests for assessing the significance of the
difference in support for two topologies (e.g., the "best" topology in
which a group is monophyletic vs. the "best" topology in which it is not).
These tests essentially put confidence limits on the two solutions and ask
if those limits overlap.  One such nonparametric test is Templeton's "T"
test (Templeton 1983. Evolution 37:221; modified by Larson 1994 in
Shierwater et al., eds. Molecular Ecoloty and Evolution: Approaches and
Applications. Birkhauser) that uses Wilcoxon ranked sums to decide whether
the data are significantly more parsimonious on one tree than another.
Another test is the Kishino-Hasegawa parametric test (1989. J. of Molec.
Evol. 29:170) that tries to get at the same thing by assuming a parametric
distribution of character-by-character length/likelihood differences that
is centered on 0 for trees that are not significantly different.  A
variation on Faith's permutation test (implemented in PAUP as "compare 2
trees"), cited above, contrasts the difference in any two topologies in
much the same way; again, it can be made more stringent by making it
two-tailed.  Recently Huelsenbeck, Hillis, and Jones (in Ferraris and
Palumbi, eds. Molecular Zoology.  1996.  Wiley.) have proposed a
"parametric bootstrap" under the likelihood framework in which a null
distribution of ranomized data sets is produced in which the "observed"
model and model parameters (i.e., the model and parameters inferred from
the observed data) are held constant.  This distribution is then used to
assess the significance of various results inferred from the observed data.


A comment: There is probably a compromise between affording all possible
trees equal a priori probability vs. affording a single (most
parsimonious/most likely) tree all of the posterior probability.  For
instance, every character implies a tree.  Trees implied by a single
character are usually largely unresolved when lots of taxa are involved.
If only the resolved parts of these trees are considered, we could come up
with a set of resolved trees (including all possibilities in subtrees where
no character affords resolution, or in which characters conflict about
topology) that is implied by all the characters considered individually.
Add to this subset all possible trees that can be generated by the
Archie/Faith permutation method (which probably includes the first set; I'd
have to think about that) and all trees that have ever been proposed by any
taxonomist, and you will have a subset of all possible trees where,
arguably, the majority of prior probability should reside.  Now perform
analyses and let the data suggest how the probability should be distributed
a posteriori.

>Zander:
>The frequentist statistical explanation is copasetic. So is the
>Bayesian, but we must remember that we are assigning probabilities to
>one event that happened in the past, one throw of the dice (worse, a
>concatenated series of throws of the dice). So here we go with
>statistical relevance again. An increase in probability does not
>necessarily confer on a theory more evidence for than against. In
>medicine, an increase in probability of disease is cause for concern but
>I for one would like to see *real good* confidence in a null, not just
>an increase in probability. How is your confidence measured?
>

"Probability" in the context of phylogeny reconstruction cannot mean how
close we are to obtaining the "true" tree.  It seems like, instead, it is a
measure of error + belief.  Under certain assumptions, the probability can
tell us how likely it is that a given result is due to random (sampling or
observation) error.  Under other assumptions, it can tell us how much we
ought to alter our prior belief in something given new data and given the
methods by which we extract patterns from those data.  To the extent that
patterns can provide reliable clues about the processes that generated
them, and to the extent that our methods allow us to distinguish among
those patterns, the probabilities might be telling us something about how
well we have reconstructed phylogeny, but that is pretty far removed.  For
example, we expect that only correlation due to phylogeny will produce
congruent patterns across diverse character systems.  We go out and we
measure the amount of congruence and compare it to some null distribution
that we expect due to chance and/or to some other distribution that we
expect from alternative processes (say, your example of convergence due to
parallel selection) and we draw a conclusion.  Statistics can tell us how
far our observations depart from the alternative expectations, once we have
defined them.  Whether this departure corresponds to our confidence that we
have accurately reconstructed phylogeny depends on whether we've covered
all possible sources of patterning and whether we've accurately modeled the
kinds of patterns they'd produce . . . quite a stretch.  So much of a
stretch, in fact, that our conclusion is undoubtedly sensitive to Bayesian
prior belief based on "background knowledge."

>>
>I must not be reading the right journals. Would you kindly cite me one
>or two papers that come up with a tree that is not obvious but which
>shows how massive congruence of lots of data from different systems has
>solved a problem in phylogeny? This is where we should be traveling, but
>along the way I see too many claims of having got there already.

I will readily agree with you that there are too many papers using scanty
(usually molecular) data to superficially "overturn" traditional systematic
groupings.  However, one series of studies that comes to mind in which a
group of researchers has recovered largely congruent phylogenies from
morphology and multiple genes is represented most recently by Fang et al.
1997.  Systematic Biology 46:269.  Regier, Mitter, Poole, and various grad
students and postdocs have explored particular taxa and character systems
deeply.  This contrasts markedly with the more common approach of using
whatever genes (primers) are available, producing a study, and moving on,
and is much more likely to benefit phylogenetics in the long run.

___________________________________
Ted Schultz, Research Entomologist
Department of Entomology, MRC 165
National Museum of Natural History
Smithsonian Institution
Washington, DC 20560
U.S.A.

schultz at onyx.si.edu
Phone (voice and fax): 202-357-1311