An exact adaptive test with superior design sensitivity in an observational study of treatments for ovarian cancer

A sensitivity analysis in an observational study determines the magnitude of bias from nonrandom treatment assignment that would need to be present to alter the qualitative conclusions of a na\"{\i}ve analysis that presumes all biases were removed by matching or by other analytic adjustments. The power of a sensitivity analysis and the design sensitivity anticipate the outcome of a sensitivity analysis under an assumed model for the generation of the data. It is known that the power of a sensitivity analysis is affected by the choice of test statistic, and, in particular, that a statistic with good Pitman efficiency in a randomized experiment, such as Wilcoxon's signed rank statistic, may have low power in a sensitivity analysis and low design sensitivity when compared to other statistics. For instance, for an additive treatment effect and errors that are Normal or logistic or $t$-distributed with 3 degrees of freedom, Brown's combined quantile average test has Pitman efficiency close to that of Wilcoxon's test but has higher power in a sensitivity analysis, while a version of Noether's test has poor Pitman efficiency in a randomized experiment but much higher design sensitivity so it is vastly more powerful than Wilcoxon's statistic in a sensitivity analysis if the sample size is sufficiently large.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS508 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Error-free milestones in error prone measurements

A predictor variable or dose that is measured with substantial error may possess an error-free milestone, such that it is known with negligible error whether the value of the variable is to the left or right of the milestone. Such a milestone provides a basis for estimating a linear relationship between the true but unknown value of the error-free predictor and an outcome, because the milestone creates a strong and valid instrumental variable. The inferences are nonparametric and robust, and in the simplest cases, they are exact and distribution free. We also consider multiple milestones for a single predictor and milestones for several predictors whose partial slopes are estimated simultaneously. Examples are drawn from the Wisconsin Longitudinal Study, in which a BA degree acts as a milestone for sixteen years of education, and the binary indicator of military service acts as a milestone for years of service.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS233 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Cross-screening in observational studies that test many hypotheses

We discuss observational studies that test many causal hypotheses, either hypotheses about many outcomes or many treatments. To be credible an observational study that tests many causal hypotheses must demonstrate that its conclusions are neither artifacts of multiple testing nor of small biases from nonrandom treatment assignment. In a sense that needs to be defined carefully, hidden within a sensitivity analysis for nonrandom assignment is an enormous correction for multiple testing: in the absence of bias, it is extremely improbable that multiple testing alone would create an association insensitive to moderate biases. We propose a new strategy called "cross-screening", different from but motivated by recent work of Bogomolov and Heller on replicability. Cross-screening splits the data in half at random, uses the first half to plan a study carried out on the second half, then uses the second half to plan a study carried out on the first half, and reports the more favorable conclusions of the two studies correcting using the Bonferroni inequality for having done two studies. If the two studies happen to concur, then they achieve Bogomolov-Heller replicability; however, importantly, replicability is not required for strong control of the family-wise error rate, and either study alone suffices for firm conclusions. In randomized studies with a few hypotheses, cross-split screening is not an attractive method when compared with conventional methods of multiplicity control, but it can become attractive when hundreds or thousands of hypotheses are subjected to sensitivity analyses in an observational study. We illustrate the technique by comparing 46 biomarkers in individuals who consume large quantities of fish versus little or no fish.Comment: 33 pages, 2 figures, 5 table

Isolation in the construction of natural experiments

A natural experiment is a type of observational study in which treatment assignment, though not randomized by the investigator, is plausibly close to random. A process that assigns treatments in a highly nonrandom, inequitable manner may, in rare and brief moments, assign aspects of treatments at random or nearly so. Isolating those moments and aspects may extract a natural experiment from a setting in which treatment assignment is otherwise quite biased, far from random. Isolation is a tool that focuses on those rare, brief instances, extracting a small natural experiment from otherwise useless data. We discuss the theory behind isolation and illustrate its use in a reanalysis of a well-known study of the effects of fertility on workforce participation. Whether a woman becomes pregnant at a certain moment in her life and whether she brings that pregnancy to term may reflect her aspirations for family, education and career, the degree of control she exerts over her fertility, and the quality of her relationship with the father; moreover, these aspirations and relationships are unlikely to be recorded with precision in surveys and censuses, and they may confound studies of workforce participation. However, given that a women is pregnant and will bring the pregnancy to term, whether she will have twins or a single child is, to a large extent, simply luck. Given that a woman is pregnant at a certain moment, the differential comparison of two types of pregnancies on workforce participation, twins or a single child, may be close to randomized, not biased by unmeasured aspirations. In this comparison, we find in our case study that mothers of twins had more children but only slightly reduced workforce participation, approximately 5% less time at work for an additional child.Comment: Published in at http://dx.doi.org/10.1214/14-AOAS770 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Comment: The Place of Death in the Quality of Life

Comment on The Place of Death in the Quality of Life [math.ST/0612783]Comment: Published at http://dx.doi.org/10.1214/088342306000000277 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Covariance Adjustment in Randomized Experiments and Observational Studies

By slightly reframing the concept of covariance adjustment in randomized experiments, a method of exact permutation inference is derived that is entirely free of distributional assumptions and uses the random assignment of treatments as the reasoned basis for inference.\u27\u27 This method of exact permutation inference may be used with many forms of covariance adjustment, including robust regression and locally weighted smoothers. The method is then generalized to observational studies where treatments were not randomly assigned, so that sensitivity to hidden biases must be examined. Adjustments using an instrumental variable are also discussed. The methods are illustrated using data from two observational studies

Some Counterclaims Undermine Themselves in Observational Studies

Claims based on observational studies that a treatment has certain e§ects are often met with counterclaims asserting that the treatment is entirely without e§ect, that all associations with treatment are produced by biased treatment assignment. Some counterclaims undermine themselves in the following speciÖc sense: presuming the counterclaim to be true may strengthen the support that the original data provide for the original claim, so that the counterclaim fails in its role as a critique of the original claim. In mathematics, a proof by contradiction supposes a proposition to be true en route to proving that the proposition is false. Analogously, the supposition that a particular counterclaim is true may justify an otherwise unjustiÖed statistical analysis, and this added analysis may interpret the original data as providing even stronger support for the original claim. More precisely, the original study is sensitive to unmeasured biases of a particular magnitude, , but an analysis that supposes the counterclaim to be true may be insensitive to much larger unmeasured biases, 0 \u3e . Illustrated using data from the US Fatal Accident Reporting System

E-Learning. A study of students’ attitudes and learning outcome when using blended learning with integration of multimedia instructions

The advent of new technologies has provided opportunities and challenges for education institutions to seek more effective ways of teaching and learning. Elearning is now an established phenomenon in education and institutions are increasing their effort to offer greater flexibility, more personalized learning, and greater learner satisfaction. Consequently blended learning has emerged as a delivery method that addresses the face-to-face advantages of a traditional classroom and the time efficiency and location convenience of an online teaching and learning environment. The opportunities of flexibility and convenience are not evident in a classroom environment. However the face-to-face interactions provide the foundation for social communication which can be critical to online learning. Educators who are comfortable with traditional classroom delivery will meet learners’ enhanced demands for flexibility through online presence for courses. As educators are generally time-pore, and with little clear direction, research can give them valuable insights into advantaged and disadvantaged of various teaching and learning formats. This thesis examines students’ attitudes when using blended learning with integration of multimedia instructions. It identifies aspects around cognitive learning outcome and provides insight into students’ experiences and their overall satisfaction with this instructional design. The main objective with this initiative is to sustain the evolution from traditional teaching to active learning and to better integrate the increasing number of educational resources online. In particular this study includes aspects about students’ attitudes towards using a blended learning format, learning theories, the instructional principles of multimedia production, and identification of optimal ways to use e-learning. This thesis contributes to the field of e-learning by three main contributions (A1-A3): A1: A contribution utilizing blended learning with integration of multimedia instructions. The quality of the blended learning format is discussed on the basis of the attitudes and experiences from the adult participants. The contribution outlines characteristics about good properties of multimedia instructions to supplement traditional classroom teaching. The content of A1 constitutes a paper that is submitted for journal publication. A2: A model for testing the cognitive learning outcome using a blended learning format with two different teaching treatments. A2 constituted the main part of a conference proceedings paper. The study further contributes in a book with the title: Cases on managing e-learning: Development and implementation. Will be released in 2012. A3: This contribution outlines a blended learning course design for postgraduate dental students with emphasis on flexibility and location convenience. The course was redesigned from a former traditional course format. The content of A3 constitutes a paper that is submitted for journal publication. The research makes a contribution in the exploration of the advantages and disadvantaged of utilizing blended learning. The research methods comprise both quantitative and qualitative investigation approaches. The empirical data for this thesis were collected through 149 participating students and 13 semi-structured interviews. The thesis supports the view of increased favorable ways of teaching and learning when using new online technologies. However no evidence for increased cognitive learning outcome was identified. Nonetheless the blended learning format with integration of multimedia instructions holds an experienced potential for improved quality of teaching and learning in terms of enhanced satisfaction among learners. The main findings. The thesis contributes to the field of e-learning by the following main contributions. First, an identification of educational key issues favourable to a blended learning format with integration of multimedia instructions; second, identification of factors to produce high quality multimedia instructions; third, the design of a test procedure to conduct measurements on cognitive learning outcome based on a basic retention level and a more advanced transfer level; forth, suggestions how to improve the influence of the internet media for future postgraduate dental educational programs