Member-only story
The O’Brien-Fleming Boundary: Interim Checks of Experiment Design, Part 2
How O’Brien-Fleming deviates from Pocock blocking and effectively stops an experiment to save time, money and possibly lives.
Experiment design and analysis is an interesting career avenue for the aspiring statistician/programmer. If you can construct a sound hypothesis test, then you can simultaneously help people and earn cash. Experiments can be complex and costly. If a research institution staff can save money by stopping a test when the results are relatively obvious, it is a good thing for that institution. It may also mean that an effective experimental treatment will be made widely available sooner. You can help facilitate that by building the skill to determine if an “early stop” is the smart move in a research study. There are a few methods to master that can provide that information and one of them is O’Brien-Fleming boundary determination.
The O’Brien-Fleming boundary is a determination of when to stop an experiment. It provides a level of detail that makes it possible to halt exactly at the point when outcomes become fairly obvious. It was proposed as a multiple testing procedure for comparing two treatments when response to treatment is both dichotomous (i.e., success or failure) and immediate. The test statistic for each test is the ordinarily accepted (Pearson) chi-square statistic based on all data collected to that point.
The maximum number of tests and the number of observations collected between successive tests is fixed in advance. The overall size of the procedure can be controlled with virtually the same accuracy as the single sample chi-square test based on the maximum number of tests times the sum of the number of observations. The power, or the probability that the test will reject a false null hypothesis, is comparable to cases when the O’Brien-Fleming boundary is not used. This is powerful, indeed.
The proper utilization of this element in experiment design affords the invaluable opportunity to terminate early when one treatment performs markedly better than the other. A multiple testing procedure may thus eliminate potential cost and ethical problems that often accompany clinical trials.
