Measuring Teacher Conscientiousness and its Impact on Students: Insight from the Measures of Effective Teaching Longitudinal Database

Although research has been unable to find strong links between observable teacher characteristics and a teacher’s ability to improve student achievement, it has generally not considered the role that teacher non-cognitive skills play in affecting student outcomes. In this article, we validate several novel performance-task measures of teacher conscientiousness based upon the effort that teachers exert completing a survey and use these measures to examine the role that teacher conscientiousness plays in affecting both student test scores and student non-cognitive skills. We conduct our analysis using the Measure of Effective Teaching Longitudinal Database where teachers were randomly assigned to their classrooms in the second year of the study. We exploit this random assignment to estimate causal impacts of teachers on their students’ outcomes during the second year of the MET project. We find that our survey-effort measures of teacher conscientiousness capture important dimensions of teacher quality. More conscientious teachers are more effective at improving their student conscientiousness but not their student test scores. Additional analysis suggests that traditional measures of teacher quality largely fail to capture a teacher’s ability to improve student conscientiousness, though measures of teacher quality based upon student ratings and one particular classroom observation protocol are exceptions

Extent, Correlates, and Consequences of Careless and Inattentive Responding in Certification Job Analysis Surveys

Survey data quality is influenced by the care and attention that respondents take in answering questions. Careless and inattentive (CI) responding is a confound in survey data that can distort findings and lead to incorrect conclusions. This quantitative study explored CI responding in job analysis studies supporting occupational certification programs and its relationship to survey features, data quality measures, and test content validity. Satisficing theory served as the framework, and secondary analysis of 3 job analysis surveys was undertaken. Results indicated that 9-33% of respondents engaged in CI responding, with the rate differing by CI index used (Mahalanobis distance, long string analysis, or person-total correlation) and by occupation. Each index detected a distinct pattern of carelessness, supporting the use of multiple indices. The indices performed best detecting carelessness in frequency ratings and may not be useful for all job analysis rating scales. Partial support was found for relationships between carelessness and survey features. CI responding had a minimal impact on mean ratings, correlations, and interrater reliability, and had no impact on certification test content outlines. By providing guidance and caution on the use of CI response detection methods with job analysis survey data, this study produced two potential avenues for social change. For practitioners conducting occupational job analyses, the use of CI detection methods can enhance the validity of data used to make certification decisions. For researchers, follow-up studies can yield a more nuanced understanding of the most appropriate use of these methods in the job analysis context

Public Administrator Aversion to Randomized Controlled Trials

This dissertation examines whether public administrators are averse to randomized controlled trials, and why. More specifically, it explores whether public administrators are reluctant or opposed to conducting RCTs for the evaluation of social welfare programs in situations where they would help to answer an important and useful policy question and are feasible to conduct. While RCTs are widely used to evaluate clinical interventions and practices, they are used far less often in social policy. I first propose a conceptual framework for understanding the public administrator’s decision-making process as they contemplate whether to conduct an RCT, outlining nine factors that likely contribute to RCT aversion. Then, using a survey experiment with a nationally representative sample, I investigate three research questions. First, do people, on average, prefer a quasi-experiment to the RCT? Second, do features of the policy environment that create a greater perceived difference in treatment between groups contribute to more RCT aversion? And third, do preferences for the RCT differ for public administrators compared to their non-public administrator peers? I find that the majority of people demonstrate a strong preference for the quasi-experiment to the RCT. Public administrators are RCT averse on average, but less so compared to their general public peers. Additionally, I find that public administrators are likely to be more RCT averse when the intervention is perceived to be very different, and potentially superior, to the status-quo option available to members of the control group. People who are not public administrators are not sensitive to features of the policy environment. I conclude by outlining several avenues for future exploration of public administrator RCT aversion, and implications for social policy researchers, evidence-based policy advocates, and public policy and administration educators

Validity of the overclaiming technique as a method to account for response bias in self-assessment questions : analysis on the basis of the PISA 2012 data

The presented work is devoted to study the validity of overclaiming technique (OCT) as a measure of response (positivity) bias. Three main aims of the analyses performed were: a) assess methods' utility to enhance predictive validity of self-report by accounting for response biases, b) investigate proposed mechanisms of overclaiming, c) expand nomological network of the method by presenting a wide set of both individual-level and cluster-level (school) correlates. The obtained results pointed that OCT can be used in order to account for response biases in self-report data. Important differences regarding use and interpretation of the different OCT scoring systems were found and commented. Two systems, one based of signal detection theory (SDT), other on item response theory model (IRT), were proposed as viable scorings of OCT. Choice between them is not trivial as it influences results' interpretation and model specification. Three possible mechanisms of overclaiming were tested: a) motivated response bias (self-favouring bias, socially desirable responding), b) memory bias (overgeneralised knowledge or faulty memory control) and c) response styles and careless responding. The results pointed that all three mechanisms are probable and that overclaiming is most probably a heterogenous phenomenon of multiple causes. However, the analyses pointed out that one of the memory bias hypotheses, the overgeneralised knowledge account, does not hold and that there is much more evidence for the competitive metacognitive account. It is to said that overclaiming is at least partially attributable to insufficient monitoring of one's knowledge. Evidence for a relation between careless responding and overclaiming was also obtained, indicating that at least some of the overclaimed responses can be attributed due to inattentive responding. Obtained results on the relations between response styles and overclaiming were complicated; they warrant further studies as the results here probably greatly depend on the technical details of analysis, e.g. response style definition and coding adopted. The analysed cluster-level covariates demonstrated that only very limited portion of OCT variance can be ascribed to the school-level of analysis. Gender, socio-economic status and locus of control proved to be significantly related to overclaiming among the individual-level correlates assessed. Boys yielded higher overclaiming bias than girls and students of external locus of control were more biased in their self-reports in comparison to students of internal locus of control. The work comprises also analysis of the PISA's OCT latent structure. The results evidenced bifactor structure of the scale, with the general factor interpreted as math ability while the two specific factors were given a tentative explanation concentrated around item difficulty (one specific factor emerged for easy items, one for hard items). These findings point to a multi-dimensional character of OCT and a large role played by domain ability in OCT responding. Moreover, latent class analysis (LCA) performed identified an "overclaiming" group among the participants which was characterised by high overclaiming and unwarrantedly high self-report profile regarding math-related abilities and social life. However, this group counted only around 9% of the total sample. Implications of these findings are commented in the work, along with theoretical integration and ideas for future studies with the use of OCT

Watch Me Give: Narcissism as a Moderator to Donating to a Nonprofit

With increased online connectivity in the current generation, more non-for-profit organizations are focusing their efforts on producing online marketing appeals. It has, therefore, become necessary to identify the effect some appeals have on different people. The present study seeks to determine whether an organization offering increased recognition will result in a greater willingness to give by people with narcissistic personalities. Additionally, it will determine if “willingness” to give or “amount” given are different between genders or employment status, as prior research suggests. Results indicated that whether the organization offered increased recognition or not, narcissism did not affect willingness to give or amount given. Women were more willing to give than men and employment status did not hold any bearing on the willingness to give or amount given. Limitations of the study are provided and implications for future research are discussed

COVID-Dynamic: A Large-Scale Longitudinal Study of Socioemotional and Behavioral Change Across the Pandemic

The COVID-19 pandemic has caused enormous societal upheaval globally. In the US, beyond the devastating toll on life and health, it triggered an economic shock unseen since the great depression and laid bare preexisting societal inequities. The full impacts of these personal, social, economic, and public-health challenges will not be known for years. To minimize societal costs and ensure future preparedness, it is critical to record the psychological and social experiences of individuals during such periods of high societal volatility. Here, we introduce, describe, and assess the COVID-Dynamic dataset, a within-participant longitudinal study conducted from April 2020 through January 2021, that captures the COVID-19 pandemic experiences of \u3e1000 US residents. Each of 16 timepoints combines standard psychological assessments with novel surveys of emotion, social/political/moral attitudes, COVID-19-related behaviors, tasks assessing implicit attitudes and social decision-making, and external data to contextualize participants’ responses. This dataset is a resource for researchers interested in COVID-19-specific questions and basic psychological phenomena, as well as clinicians and policy-makers looking to mitigate the effects of future calamities

Investigating the Efficacy of Novel Measures of Careless Responding to Tests

Research has demonstrated that careless responding (CR) threatens the construct validity of measures (see Huang et al., 2015; Wise & Kong, 2005). Researchers have developed and studied many measurement approaches to capture CR in surveys, with different survey measures compensating for the practical or empirical limitations of other measures. This research is distinguished from ability test CR research because ability tests are fundamentally different from surveys. Within ability tests, CR research has focused only on response time and self-report measures of CR, both of which carry limitations. The former is inflexible because the index necessitates item-level response time information, and therefore cannot be used in pen-and-paper tests or online tests without such item-level information available. The latter index is plagued by theoretical and empirical shortcomings. Thus, the purpose of my study is to find a comparably valid and more flexible approach through testing the efficacy of five survey CR measurement approaches, namely the infrequency approach, instructed-response approach, the consistency approach, the self-report approach, and long-string analysis, in capturing CR in tests. In a sample of 291 undergraduate students, I found strong support for using the infrequency approach to assess careless responding, weak support for the instructed-response approach and long-string analysis, and no support for the self-report or consistency approaches

The Use of Online Panel Data in Management Research: A Review and Recommendations

Management scholars have long depended on convenience samples to conduct research involving human participants. However, the past decade has seen an emergence of a new convenience sample: online panels and online panel participants. The data these participants provide—online panel data (OPD)—has been embraced by many management scholars owing to the numerous benefits it provides over “traditional” convenience samples. Despite those advantages, OPD has not been warmly received by all. Currently, there is a divide in the field over the appropriateness of OPD in management scholarship. Our review takes aim at the divide with the goal of providing a common understanding of OPD and its utility and providing recommendations regarding when and how to use OPD and how and where to publish it. To accomplish these goals, we inventoried and reviewed OPD use across 13 management journals spanning 2006 to 2017. Our search resulted in 804 OPD-based studies across 439 articles. Notably, our search also identified 26 online panel platforms (“brokers”) used to connect researchers with online panel participants. Importantly, we offer specific guidance to authors, reviewers, and editors, having implications for both micro and macro management scholars

Teachers and the Development of Student Noncognitive Skills

Scholars of education policy are increasingly aware of the independent role that noncognitive skills (e.g., self-regulation, Social skills, and other personality or character traits) play in long- and short-run student well-being. However, little is known about how these skills are effectively developed. One theory is that noncognitive skills are developed through role modeling by teachers. A student, by virtue of observing and sharing a Social connection with his or her schoolteachers, begins to emulate noncognitive skills that they exhibit. In this dissertation, I test this theory. I focus specifically on noncognitive skills related to conscientiousness and measure them using new behavioral proxies based upon survey effort. In chapter 2, I use panel data and techniques to show that students experience declines in conscientiousness in years when they are taught by teachers who exhibit less conscientiousness. Assuming that students are not systematically sorted to teachers with varying degrees of conscientiousness based upon time-varying student characteristics, the relationship between student and teacher conscientiousness can be interpreted as causal. In chapter 3, I corroborate these findings in an analysis that possesses greater internal validity. Using data in which teachers were randomly assigned to classrooms, I find that students become more conscientious when they are taught by more conscientious teachers. If teacher noncognitive skills are transmitted to students, particularly by role modeling, it may be desirable for school leaders to ensure that their school communities exhibit coherence over a values system that they desire students to embrace. In chapter 4, I discuss a policy proposal for this end. Specifically, I test whether school leaders who possess more flexibility in hiring and dismissing teachers are more adept at building a staff that embodies a core set of values. I provide descriptive evidence suggesting that schools exhibit more values coherence when principals have more autonomy over personnel decisions. In conclusion, education policy has only recently given increasing attention to noncognitive skills. More effort must be devoted to studying this topic as carries normative implications not only for the means and aims of existing educational institutions but also for the labor of reforming and improving them

Identification and Compensation of Aberrant Response Patterns: Quality of Comfort Assessments in Human Subject Trials

Whenever human samples are used to assess a set of different items by using self-reports, concerns regarding the quality of data can arise because participants potentially reply without paying sufficient attention to the contents. The present study reviews different indices of aberrant response patterns and investigates this phenomenon by applying some indicatorson subjective assessments of thermal comfort of N = 160 subjects within the DLR-project Next Generation Train (NGT). Based on this approach, a full sample was compared with a cleaned sample to examine whether aberrant responses have significantly biased the results. Overall, the differences were negligible, which is a proof for the data quality and the utility of human subject trials in comfort assessments. However, the usefulness of indices of aberrant response behaviour is still demonstrated and further suggestions to prevent and reduce aberrant responses in similar studies are discussed