Date & Time: Wednesday, March 12, 2014, 15:00-16:00.
Venue: A1A2, CDEEP
Speaker: Shyamal Krishna De, Binghamton University
Title: Controlling Error Rates for Multiple Hypothesis Testing in Sequential Experiments
Abstract: A number of sequential experiments such as sequential clinical trials with multiple endpoints, multichannel change-point detection problems, etc., involve multiple hypothesis testing. For instance, in a clinical trial, patients are typically collected sequentially to answer multiple questions about safety and efficacy of a new treatment based on multiple endpoints. That is, a family of hypotheses is tested simultaneously based on these endpoints. A separate decision, accept or reject, is expected on each individual hypothesis. In this work, we develop stopping rules and decision rules such that desired error rates are controlled at pre-specified levels and the expected sample size is as low as possible.
In case of simultaneous testing of a large number of hypotheses (e.g., in genetics), a few Type I and Type II errors can often be tolerated. For such applications, many author consider controlling generalized family wise error rates GFWER-I and GFWER-II defined as the probabilities of making at least $ k (> 1) Type I errors and at least $ m (> 1) $ Type II errors respectively. Fixed-sample multiple test procedures cannot control GFWER-I and II simultaneously. Our proposed procedures control both GFWER-I and II at pre-specified levels under any combination of true null and alternative hypotheses.
For large-scale multiple testing, False Discovery Proportion (FDP) and False Non-discovery Proportion (FNP) are also popular error rates. FDP (FNP) is defined as the number of false rejections (false acceptances) of null divided by the total number of rejections (acceptances) of null. The theory and methodology for controlling FDP and FNP are not developed yet for sequential experiments. We propose a test procedure that controls the tail probabilities of both FDP and FNP at some prescribed levels. For testing simple versus simple hypothesis and for certain composite hypotheses, we provide a mathematical proof that the proposed method controls the probabilities of FDP and FNP being more than $ g1 $ and $ g2 $ respectively, where $ g1 $ and $ g2 $ are some fixed numbers between $ 0 $ and $ 1 $. Moreover, we show that the well-known FDR and FNR, defined as the expected values of FDP and FNP, can also be controlled simultaneously using the proposed procedure.
Key Words: Expected Sample Size; False Discovery and Non-discovery Proportions; GFWER-I and II; Multiple Hypothesis Testing; Sequential Test; Stopping Rule