Statistical power of KH and SH tests?

[Laurent at]

Peter -

> Ideally, you want both alpha and beta to be low.

With everything else fixed, beta decreases with the degree of falseness of
your null hypothesis, which you usually don't know (unless you are making a
simulation). In a real test, there won't be a single value for beta, but a
beta function.
In some cases, it is possible to compute this function analytically. For
example, in the simple case of testing the equality of two means, it is
relatively straightforward to express beta as a function of the (unknown)
actual difference between these two means. As beta is the probability that
you don't detect an existing difference, it will quite logically be high if
the existing difference is small, and low if this difference is large.


> However,
> if beta is sufficiently low, when you fail to reject the null
> hypothesis, you can actually go so far as to accept the null
> hypothesis, since the chance of Type II error is sufficiently low.

I think there's a problem with the way you express your null.
Null hypotheses usually don't sound like "A and B are not significantly
different", but rather like "A and B are identical"...which is why you can't
accept them.
If you (or your test) can detect a clear difference between A and B, you
will reject the null hypothesis. But when you can't see a difference, this
does not (never!) mean that there is none... Possibly the difference is
there, but it's just too small and you didn't look hard enough ; it's
always possible to look harder.

What you can do, of course, is to check the values of beta for departures
from the null hypothesis small enough that you would consider them
unimportant in the context of the problem you are studying. Then you could
possibly accept that A and B, if they differ, do it "negligibly". But this
would not involve accepting the null hypothesis.


Regarding your original question, though I'm afraid I'm very unfamiliar with
the practice of KH or SH tests, you might perhaps find the following paper
interesting :

Aris-Brosou S. (2003) : Least and most powerful phylogenetic tests to
elucidate the origin of the seed plants in presence of conflicting signals
under misspecified models. Syst. Biol. 52 (6) : 781 - 793.

In case you don't have access to this journal, a preprint can be downloaded
from here :
http://statgen.ncsu.edu/stephane/papers.htm


'Hope this is of some help...
Cheers,
Laurent


Laurent Raty
l.raty at skynet.be
Brussels, Belgium