\magnification=1200
\baselineskip=20pt
\nopagenumbers
\font\big=cmr12 scaled \magstep2

\centerline{\bf STANFORD UNIVERSITY}
\centerline{\bf DEPARTMENT OF STATISTICS}
\centerline{\big DEPARTMENT SEMINAR}
\bigskip

\baselineskip=12pt
\centerline{4:15 p.m., Tuesday, May 28, 2002}
\centerline{Sequoia Hall Room 200}
\centerline{(Cookies at 3:45 in 1st Floor Lounge)}

\bigskip
\baselineskip=15pt
\centerline{\sl Jiashun Jin}
\centerline{\sl Statistics Department}
\centerline{\sl Stanford University}
\bigskip
\centerline{\bf Optimal Adaptivity of the Higher Criticism}
\bigskip

Higher Criticism, or second-level significance testing,
is a multiple comparisons concept mentioned in passing
by Tukey (1976). It concerns a situation where there are many independent
and unrelated tests of significance. Tukey suggested to compare the
fraction of observed significances at a given $\alpha$-level to the
expected fraction under the joint null, in fact he suggested to
standardize the difference of the two quantities and form a z-score;
the resulting z-score tests the significance {\it of the body of
significance tests}. We consider a generalization, where we maximize this
z-score over a range of significance levels $0 < \alpha < \alpha_0$. We
are able to show that the resulting {\it Higher Criticism
statistic} is
effective at resolving a very subtle testing problem where we are testing
whether $N$ normal means are all zero versus the alternative that a very
small fraction is nonzero. The subtlety  of this `sparse normal means'
testing problem can be seen from work of Ingster (1999) and Jin(2002), who
studied such problems in great detail. In their studies, they identified
an interesting range of cases where the small fraction of nonzero means is
so small that the alternative hypothesis exhibits little noticeable effect
on the distribution on the $p$-values either for the bulk of the tests or
for the few most highly significant tests. Nevertheless, the fraction of
nonzero means is delicately calibrated so that the likelihood ratio test
for a precisely-specified alternative would still succeed in separating
the two hypotheses.

We show that the higher criticism is successful throughout the same region
where the precisely specified likelihood ratio test would succeed, and
thus, higher criticism is in a sense optimally adaptive to unknown
sparsity and size of the non-null effects. While our theoretical work is
largely asymptotic, we provide simulations in finite samples, and suggest
some possible applications.


\bye