Investigating the Misrepresentation of Statistical Significance in Empirical Articles

In an attempt to preserve research integrity, the aim of this study is to examine how often statistical results are being misrepresented in empirical studies by using terms such as “marginally significant,” “approached significance,” or “trend toward significance” when interpreting findings. The use of these terms gives ambiguous significance to results that are in fact nonsignificant, which threatens future research by contributing to issues such as the replication crisis. For this study, data were coded from 437 empirical articles published online in The Journal of Personality and Social Psychology (JPSP) over a 4-year period between 2017 and 2020. According to our findings, although misrepresentation of statistical results are prevalent within JPSP articles, rates decreased significantly over the four-year time period examined. Additionally, as the number of studies published in JPSP increased each year during the four-year period examined, there may be a potential rise in representatively sound studies and decrease of misrepresentation within this discipline

The Effects of Self-Care on Undergraduate Stress

Previous research shows that excessive stress can have a significant, negative effect on one’s overall cognitive efficiency and that stress is negatively correlated with self-care routines. The present research builds upon this body of knowledge by gathering data from an undergraduate sample (N = 200) with 44 males and 156 females (MAge = 21.22). Participants’ stress and self-care practices were measured at weeks 3, 6, 9, 12, and 15 of their semester using the Depression Anxiety Stress Scale and the Mindful Self-Care Scale. We hypothesized that increased stress would result in decreased self-care practices and that predisposed self-care at time 1 could be used to predict stress levels at times 3 and 5. A cross-lagged panel analysis supported this hypothesis, indicating simultaneously that self-care was significantly correlated with stress and that the two factors were significantly predictive of one another at later time points

Assessing Data Quality Across the Semester from an Undergraduate Psychology Participant Pool

When collecting data for research within the psychological sciences, it is common for academic institutions to rely upon the undergraduate participant pool, often those enrolled in intro-level psych courses. Many researchers are concerned about the quality of data as it is collected across the semester. Researchers assume that data quality is worse at the end of a semester, with online studies being particularly vulnerable to this. Few studies have empirically investigated this question. To address this gap, we conducted one study across multiple semesters to assess whether data quality is worse at the end of the semester than towards the beginning. Participants signed up through Sona and completed the study in an in-person lab setting. After signing an informed consent document, participants were asked to complete several questionnaires followed by a puzzle task and were then asked to complete a few more questionnaires before being debriefed. The time it took participants to complete tasks (the consent form, questionnaires, the puzzle task, and the funnel debriefing) were all measured. Data quality was assessed by looking at the number of incorrect attention-check items, number of missed items, length of open-ended responses, response bias, answers regarding self-reported engagement, and time spent on each task as it may indicate whether participants rushed through the study or not. In the online version of this study, we found that for online studies recruiting participants from a university undergraduate pool, data quality is worse at the end of the semester than at the beginning. Participants who participated later in the semester wrote fewer words on open-ended questions, incorrectly responded to more instructed response attention check items, showed greater response bias, and reported less attention, effort, diligence, and interest. As with the online study previously conducted, we will compare data quality from those who participated at the beginning of the semester to those who participated at the end. We will also discuss how these results compare to those found in our online data quality study. We expect results in the current study to find that data quality is worse at the end of the semester than at the beginning, even in an in-person study. We also expect the data quality of the in-person study to be higher overall than the online study. This research has examined the concerns surrounding whether data quality obtained from an undergraduate participant pool decreases in quality across the semester. Although results from the online study support this idea, the effect size was small to moderate. As such, researchers should not be afraid to collect data at the end of a semester if they assess data quality. If results from the current study do not align with results from the online study or our hypotheses, it may suggest that data collected at the end of the semester for in-person studies do not decrease significantly in data quality

Data Quality: Does Time of Semester Matter?

When conducting scientific research, obtaining high-quality data is important. When collecting data from a college student participant pool, however, factors such as the time of the semester in which data are collected could cause validity issues, especially if the survey is completed in an online, non-laboratory setting. Near the end of the semester, students may experience more time pressures and constraints than at other times in the semester. These additional pressures may encourage participants to multi-task while completing the study, or to rush through the survey in order to receive credits as quickly as possible. The hypothesis of this study was that responses collected at the end of the semester would exhibit lower data quality than responses collected at the beginning of the semester. Data were collected online during the last two weeks of the fall 2018 semester (n = 312) and the first two weeks of the spring 2019 semester (n = 55). Participants were asked to write about an embarrassing situation and then completed a number of questionnaires assessing their thoughts and feelings about the event, personality traits, and participant engagement. Data quality was assessed using several different previously validated methods, including time spent on survey; the number of missed items; the number of incorrect embedded attention-check items (out of 12); the length of responses on two open-ended questions; self-reported diligence, interest, effort, attention, and whether their data should be used; and Cronbach’s alphas on the scales. Results showed that between the two groups, there were significant differences on length of open-ended responses, self-reported diligence, self-reported interest, effort, attention, neuroticism, and conscientiousness. Participants completing the study in the first two weeks of the spring 2019 semester had significantly longer open-ended responses and significantly higher levels of self-reported diligence, self-reported interest, effort, attention, neuroticism, and conscientiousness. Although there was not a significant difference in number of incorrect attention-check items between the two groups, it should be noted that only 46% of the total participants did not miss any check items. These results lend support to the hypothesis that data collected at the end of the semester may be of lower quality than data collected at the beginning of the semester. However, because the groups significantly differed on neuroticism and conscientiousness, we cannot determine whether the time of semester effect is a product of internal participant characteristics or external pressures. Nevertheless, researchers should take into account this end-of-semester data quality difference when deciding the time-frame of their data collection

Does Engagement in Online Dating Lead to Greater Dating Success for Rejection Sensitive Individuals?

Individuals high in rejection sensitivity have a lower rates of dating success and a lower probability of being in a romantic relationship, but could engaging in online dating improve the likelihood? The goal of this research is to determine whether individuals higher in rejection sensitivity indicate having more success in meeting potential romantic partners online when compared to meeting potential partners conventionally. It is hypothesized that individuals higher in rejection sensitivity will be more successful on first dates when initiated though online dating sites / apps rather than through more conventional ways of meeting potential romantic partners. An online survey was created asking participants to complete the Rejection Sensitivity Questionnaire to evaluate the level of anxiety experienced when faced with potential rejection, the Online Dating Inventory to assess engagement in online dating, and the Conventional Offline Dating Inventory, a measure created by the researchers to assess behaviors that allow the participants to meet possible romantic partners in every day life through non-internet activities. Success is operationally defined according to the goals participants had going into the date and whether or not that goal was achieved. Data collection is still ongoing; however, we expect to see significantly more success for participants higher in rejection sensitivity when initiating through online dating websites. To ensure external validity, data are being collected from two different articipation pools. Currently the survey is being administered through Sona to reach college students and these are the data that will be presented. The survey will also be uploaded to Amazon’s Mechanical Turk to reach older adults. If our hypothesis is correct, this would indicate that online dating may benefit those individuals who are high in rejection sensitivity as it could give them an alternative avenue to initiating romantic relationships and possibly finding success in those relationships

Efficacy of Guided Versus Self-Induced Learning of Web-Based Self-Compassionate Journaling by College Students

We conducted 3 studies examining the efficacy of web-based self-compassionate journaling (SCJ). The goal was to compare the effects of guided and self-administered journaling on selfreported follow-up self-compassion scores and participant perceptions of the induction. In Study 1 participants were randomly assigned to an online SCJ exercise, online narrative journaling control group, or attention control (AC) group, with groups completing tasks for 4 weeks. In Study 2 participants self-selected into either online or in-lab conditions to complete a single-time SCJ exercise. Study 3 was a replication of Study 2, with participants being randomly assigned to conditions. There were no differences in self-compassion between groups in Study 1. However, there was a small but significant overall increase in self-compassion among participants from baseline to follow-up. There were self-compassion differences between online and in-lab groups in Study 2 (with a small effect size). However, these differences were not maintained when the covariate of baseline trait self-compassion was removed from analyses. Study 3 failed to replicate significant findings from Study 2 in all but one variable: whether participants completing the induction alone found the task more difficult to complete. In general, SCJ may not be an adequate way to increase self-compassion regardless of whether it is learned online or in a laboratory setting. Furthermore, those who learn SCJ alone online report difficulty in completing the induction

An Investigation into the Structure of Self-Control

Self-control has been measured using a variety of methods including self-report measures, cognitive inhibition tasks, delay discounting and delay of gratification tasks, and persistence and willpower tasks. Although these are all theoretically linked to processes involved in self-control, recent evidence has shown that these diverse measurement techniques relate only minimally to one another. Assuming that self-control is a reflective construct, this would indicate that many of these tasks are poor indicators of self-control. The present research challenges the common assumption that self-control is a reflective construct and instead proposes that self-control is a formative construct. Conceptualizing of self-control as a formative construct could reconcile some of the inconsistencies in the literature, in particular the fact that many indicators for self-control do not correlate highly. To examine the possibility of a formative model of self-control, this research examines 13 commonly used measures of self-control and investigates indicator intercorrelations, indicator relationships with the theoretical consequences of self-control, and performs a vanishing tetrad test (Bollen & Ting, 2000). Results show that in general, indicator intercorrelations are low and nonsignificant as well as indictor correlations with theorized construct consequences. The results of the vanishing tetrad test suggest a reflective interpretation of self-control, but concerns with uniformly low covariances between indicators limit the interpretation of this test. It is concluded that currently available measures of self-control contain large sources of error variance and that questions about the ontological nature of the construct will be unanswerable until more precise measures are developed

Thinking Into the Future: How a Future Time Perspective Improves Self-Control

The dual motive model posits that self-control is the prioritization of distal motives over proximal motives when the two compete. A logical extension of this view is that any factor that increases the incentive value of a distal motive or decreases the incentive value of a proximal motive will make self-control more likely. Here it is proposed that time perspective, or an individual\u27s tendency to attend to thoughts of the past, present, or future, is one factor that influences the incentive value of competing motives. Three studies were conducted to show that time perspective influences the incentive value of competing motives, and thus influences self-control. Study 1 probes correlations and indirect effects between time perspective, incentive value, and self-control. Study 2 replicates and extends study 1 by examining additional dimensions of the future time perspective. Study 3 shows that manipulating time perspective produces changes in self-control, establishing causality. The results suggest that time perspective influences the incentive value of individuals\u27 motives and thus self-control. The results also add support to the dual motive model of self-control, since only the dual motive model predicted these relationships