Making the Error Bar Overlap Myth a Reality: Comparative Confidence Intervals

Many interpret error bars to mean that if they do not overlap the difference is statistically significant. This overlap rule is really an overlap myth; the rule does not hold true for any conventional type of error bar. There are rules of thumb for estimating P values, but it would be better to show error bars for which the overlap rule holds true. Here I explain how to calculate what I call comparative confidence intervals which, when plotted as error bars, let us judge significance based on overlap or separation. Others have published on these intervals (the mathematical basis goes back to John Tukey) but here I advertise comparative confidence intervals in the hope that more people use them. Judging statistical significance by eye would be most useful when making multiple comparisons, so I show how comparative confidence intervals can be used to illustrate the results of Tukey tests or Dunnett’s test. I also explain how to use of comparative confidence intervals to explore the effects of multiple independent variables and explore the problems posed by heterogeneity of variance and repeated measures. When families of comparative confidence intervals are plotted around means, I show how box-and-whiskers plots make it easy to judge which intervals overlap with which. Comparative confidence intervals have the potential to be used in a wide variety of circumstances, so I describe an easy way to confirm the intervals’ validity. When sample means are being compared to each other, they should be plotted with error bars that indicate comparative confidence intervals, either along with or instead of conventional error bars. This paper is based on a submission that was rejected by Psychological Methods. The original submission along with the reviewer’s comments and my responses are available at the bottom of this page

Force, Torque, and the Electromyogram: A Student Laboratory Exercise

This paper describes a muscle physiology laboratory exercise that uses electromyography to emphasize the concepts of force and torque and the role of limb position. Students perform a simple laboratory exercise that enables them to relate the angle of a joint with the concepts of force, torque, and the moment arm. Students observe the combined effects of motor unit recruitment and the variation of action potential frequency on the amount of electrical activity a muscle generates. They note that the amount of force a muscle generates also depends on motor unit recruitment and spike frequency. Here, I provide detailed protocols that students may follow. For the instructor, I provide background information, representative results, an explanation of technical matters, a list of pitfalls students may experience, and settings files for LabChart and LabScribe software. The exercise described here provides an excellent complement to, or serviceable replacement for, other muscle physiology labs while it also illustrates some principles of kinesiology

Understanding Null Hypothesis Tests, and Their Wise Use

Few students sitting in their introductory statistics class learn that they are being taught the product of a misguided effort to combine two methods of inference into one. Few students learn that many think the method they are being taught should be banned. In Understanding Null Hypothesis Tests, and Their Wise Use, I emphasize Ronald Fisher’s approach to null hypothesis testing. Fisher’s method is simple, intuitive, thoughtful, and pure. If we follow Fisher’s example, all the criticisms of null hypothesis tests melt away. Fisher on a good day. Do you collect data and then ask a friend how to analyze them? Once you have read this monograph, you will not have to do that anymore (at least not often). You will understand the concepts behind the mathematics. You will see why different types of data require different types of tests. Do you think that P is the probability of a type I error? It is not. If you fail to reject, do you accept the alternate or alternative hypothesis? You accomplish nothing by doing so. Do you equate statistical “significance” with importance? You should not. These and other misconceptions are explained and dispensed with in Understanding Null Hypothesis Tests, and Their Wise Use. After reading this monograph, you will understand why it is utter foolishness to say, We use confidence intervals instead. (Confidence intervals are wonderful, but they show the results of null hypothesis tests performed backwards.) More importantly, you will understand wise use. You will use P-values thoughtfully, not to make mindless, binary decisions. Most importantly, you will know the Big Secret that should not be a secret. A null hypothesis is infinitely precise, so many and maybe most null hypotheses cannot be correct—a fact we should know from the start. It was the legendary statistician John Tukey who explained why it is important to test such nulls anyway: to determine whether we can trust our data to tell us the direction of a difference. Getting the direction wrong is referred to as a type III or type S error. Once you have read about type S errors in Understanding Null Hypothesis Tests, and Their Wise Use, you will have a better understanding of null hypothesis tests than anyone on your block

Editorial, Your Null Hypothesis Must Be False: Test It Anyway

A historical criticism of null hypothesis testing is that a null hypothesis cannot be correct in the first place. The reason that many nulls cannot be correct is that what makes a null hypothesis a null hypothesis is its infinite precision. Only an infinitely precise hypothesis can be used to generate an infinitely precise prediction, such as t will be zero. It is around that infinitely precise prediction that we would construct a probability density function. If you are thinking, I just want to know if the difference is significant, and if you mean important, a null hypothesis test will never tell you that. Although we should know from the start when our nulls cannot be correct, there is still reason to test them: to see if we can confidently decide on the direction of a difference. In this editorial, I explore when nulls can and cannot be correct and summarize the field of directional decisions. Ronald Fisher made directional decisions. To John Tukey, the only reason to test a null was to decide on direction. And it is not a matter of using one-tailed tests. Before you submit your P values to a journal, read this editorial and rethink what you have written. The question is one of direction

Making the Error Bar Overlap Myth a Reality: Comparative Confidence Intervals

Many interpret error bars to mean that if they do not overlap the difference is statistically “significant”. This overlap rule is really an overlap myth; the rule does not hold true for any conventional type of error bar. There are rules of thumb for estimating P values, but it would be better to show error bars for which the overlap rule holds true. Here I explain how to calculate comparative confidence intervals which, when plotted as error bars, let us judge significance based on overlap or separation. Others have published on these intervals (the mathematical basis goes back to John Tukey) but here I advertise comparative confidence intervals in the hope that more people use them. Judging statistical “significance” by eye would be most useful when making multiple comparisons, so I show how comparative confidence intervals can be used to illustrate the results of Tukey test. I also explain how to use of comparative confidence intervals to illustrate the effects of multiple independent variables and explore the problems posed by heterogeneity of variance and repeated measures. When families of comparative confidence intervals are plotted around means, I show how box-and-whiskers plots make it easy to judge which intervals overlap with which. Comparative confidence intervals have the potential to be used in a wide variety of circumstances, so I describe an easy way to confirm the intervals’ validity. When sample means are being compared to each other, they should be plotted with error bars that indicate comparative confidence intervals, either along with or instead of conventional error bars

Making the Daphnia Heart Rate Lab Work: Optimizing the Use of Club Soda and Isopropyl Alcohol

Students commonly test the effects of chemical agents on the heart rate of Daphnia magna, a small crustacean. We investigated whether club soda and isopropanol are suitable test agents. Treatment groups contained 6-12 animals. Club soda caused a dose-dependent decrease in heart rate, presumably because of the anesthetic effects of CO2. Ten percent, 30%, and 50% club soda reduced mean heart rates to 78%, 57%, and 47% of initial values. The effect was transient; heart rates recovered quickly to control values even though the club soda remained present. Isopropanol’s effect was dose-dependent and sustained. Three percent, 5%, and 10% isopropanol reduced mean heart rate to 45%, 35%, and 12% of initial values. Removal of the isopropanol failed to fully reverse its effects. Ten percent isopropanol proved fatal to one animal out of the eight tested at that concentration. Both club soda and isopropanol are suitable agents for students to test. If reversibility is to be investigated, club soda should not be used. Its effects wear off even when the club soda is still present. Isopropanol is best used at 3-10% as it causes marked heart rate suppression and partial reversibility

Do Organic Solutes Trigger Particle Ingestion in the Ciliate Tetrahymena pyriformis?

Little is known regarding what triggers phagocytosis in ciliates. It is unclear as to whether these organisms will ingest inert particles without dissolved organic solutes to stimulate consumption. We fed the ciliate Tetrahymena pyriformis 3 µm polystyrene beads while the organisms were immersed in starvation medium, 0.8% glucose, and growth medium. T. pyriformis fed well in all three media. We find no evidence that Tetrahymena requires organic solutes to trigger ingestion of inert particles

Spadix Function in the Jack-in-the-Pulpit, Arisaema triphyllum

Aroids are perennial herbs characterized by inflorescences consisting of a finger-like spadix surrounded by a vase-like spathe. A prominent aroid in Georgia is the Jack-in-the-pulpit, Arisaema triphyllum. We assessed the role of the spadix in attracting insect visitors to Arisaema triphyllum. Two study sites near Dahlonega, Georgia, were chosen: one along an unnamed first-order stream and the other along third-order Cane Creek. Plants received either ablation of the distal appendix, removal of the spadix tip, or a sham ablation. Arthropod visitors were captured with a small sticky trap placed inside the spathe. Despite the treatment applied, the number of Diptera captured was not affected. In contrast, ablation reduced the number of Collembola captured to just 29 % of that of the other two treatments (interaction of taxon and treatment after square root transformation: F10,480 = 2.761, P = 0.003). Pollination in A. triphyllum has previously been attributed to fungus gnats (Diptera) and Heterothrips arisaemae (Thysanoptera). Our results suggest that Collembola, which do not fly, may play a role in pollination, perhaps within clustered plants in which long-distance travel is not necessary

Findings Consistent with Nonselective Feeding in Tetrahymena pyriformis

In amoeboid cells, food particles are engulfed only after receptors on the phagocytic cell’s membrane bind to ligands on a particle’s surface. Ciliates also feed via phagocytosis but, instead of enveloping particles, some ciliates take them up through a complex, permanent, funnel-shaped, feeding apparatus. It is unclear whether receptor-ligand interactions are needed to trigger the process. If ciliates were shown to feed selectively on certain particles over others, based on the particles’ surface properties, then receptor-ligand interactions would likely play a role in phagocytosis. The literature includes few reports of such selectivity. To further investigate this issue, we chose to study feeding preference in the ciliate Tetrahymena pyriformis. We fed Tetrahymena mixtures of orange and green, fluorescent, 3 µm, polystyrene beads at two concentrations. One of the two types of beads was coated with bovine serum albumin. Authors were blinded to experimental conditions. We found no evidence of a preference for coated or uncoated beads at either concentration. We also found no trend toward the development of selective feeding as cells acquired more beads over time. Although we cannot rule out the possibility that Tetrahymena feeds selectively, we did not find convincing evidence of such selectivity when T. pyriformis is given a choice between uncoated beads and those coated with albumin. Our results failed to demonstrate a role for molecular recognition when Tetrahymena engages in phagocytosis