Can faking be measured with dedicated validity scales? Within Subject Trifactor Mixture Modeling applied to BIDR responses

A sample of 516 participants responded to the Balanced Inventory of Desirable Responding (BIDR) under answer honest and instructed faking conditions in a within-subjects design. We analyse these data with a novel application of trifactor modeling that models the two substantive factors measured by the BIDR – Self-Deceptive Enhancement (SDE) and Impression Management (IM), condition-related common factors and item specific factors. The model permits examination of invariance and change within subjects across conditions. Participants were able to significantly increase their SDE and IM in the instructed faking condition relative to the honest response condition. Mixture modeling confirmed the existence of a theoretical two-class solution comprised of approximately two thirds of ‘compliers’ and one third of ‘non-compliers’. Factor scores had good determinacy and correlations with observed scores were near unity for continuous scoring, supporting observed score interpretations of BIDR scales in high stakes settings. Correlations were somewhat lower for the dichotomous scoring protocol. Overall, results show that the BIDR scales function similarly as measures of socially desirable functioning in low and high stakes conditions. We discuss conditions under which we expect these results will and will not generalise to other validity scales

The Effect of Grade Retention on High School Completion

Low-achieving students in many school districts are retained in a grade in order to allow them to gain the academic or social skills that teachers believe are necessary to succeed academically. This practice is highly controversial, with many researchers claiming that it leads to higher dropout rates although selection issues have complicated previous analyses. In this paper, we use a regression discontinuity design to examine the impact of grade retention on high school completion. We find that grade retention leads to a modest increase in the probability of dropping out for older students, but has no significant effect on younger students.

A Reading and Writing Placement Test: Design, Evaluation, and Analysis

Placement tests, along with the growing interest in their validation, have become increasingly important in English as a Second Language programs. To this end, the present paper illustrates procedures in designing a placement test and using it to evaluate students’ language ability by means of statistical analysis. 29 participants from three proficiency levels (beginning, intermediate, and advanced) took reading and writing placement test sections. Students’ performance at each proficiency level was analyzed separately and compared across proficiency levels. In addition, analyses of responses on a survey revealed a relationship between language learning attitudes and behaviors exhibited by the participants and their performance on the placement test. Through close examination of the process of placement test design, evaluation, and analysis, this paper provides practical guidelines for reading and writing placement testing

Theoretical and Practical Advances in Computer-based Educational Measurement

This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology

Perceptions and Cognitive Bias in Decomposition Scoring Methods in Forensic Anthropology

There is increasing recognition within the forensic science community that human examiners are prone to biases that may affect the accuracy and precision of conclusions. The analysis of decomposition to estimate the postmortem interval (PMI) is one of the important functions of forensic anthropologists, yet analytical methods that rely on the judgment of human observers, such as assessing PMI from decomposition, may be vulnerable to cognitive bias, leading to inaccurate results. No studies to date have examined the effects of cognitive bias on decomposition scoring methods. The goal of this dissertation is to understand the role of the cognitive factors of mood and motivation on the analysis of decomposition characteristics both in field and photograph contexts. Fifty undergraduate research assistants with the University of Tennessee’s Forensic Anthropology Center assessed decomposition from 10 donated individuals over several months utilizing the Total Body Score method. Four graduate students who had extensive knowledge and experience with decomposition served as experts with which to compare the accuracy of observers’ TBS values. Observers also completed two psychological measures aimed at assessing their motivations (Intrinsic Motivation Inventory) and mood (Positive and Negative Affect Schedule) at the time of scoring. It was expected that observers and experts would differ in both the field and photograph contexts and that mood and motivation factors would be predictive of TBS values. Hierarchical random intercept multiple regression models were conducted to assess the relationship between these measures of cognition and observers’ TBS scores. Observers and experts only differed in the field context possibly due to biasing contextual information regarding the placement date of the donors. No differences were found in the photograph contexts because contextual information was not available to observers. Observers in the field context were motivated by their perceived performance, how nervous or pressured they felt while completing the task, and the perceived difficulty of the task itself. Mood only played a small role in the photograph context where negative mood influenced scoring decisions. This study has implications for the application of TBS to longitudinal research conducted at outdoor decomposition facilities as well as to cross-sectional casework

Does Management Matter? Evidence from India

A long-standing question in social science is to what extent differences in management cause differences in firm performance. To investigate this we ran a management field experiment on large Indian textile firms. We provided free consulting on modern management practices to a randomly chosen set of treatment plants and compared their performance to the control plants. We find that adopting these management practices had three main effects. First, it raised average productivity by 11% through improved quality and efficiency and reduced inventory. Second, it increased decentralization of decision making, as better information flow enabled owners to delegate more decisions to middle managers. Third, it increased the use of computers, necessitated by the data collection and analysis involved in modern management. Since these practices were profitable this raises the question of why firms had not adopted these before. Our results suggest that informational barriers were a primary factor in explaining this lack of adoption. Modern management is a technology that diffuses slowly between firms, with many Indian firms initially unaware of its existence or impact. Since competition was limited by constraints on firm entry and growth, badly managed firms were not rapidly driven from the market.management, organization, IT, productivity and India

Seeking the N in LLN

Numeracy skills are a key driver of economic growth and yet, according to the Australian Bureau of Statistics (ABS 2008) nearly eight million Australian adults lack the numeracy skills to cope with everyday life and work. The vocational education and training (VET) sector is one part of the solution; however the VET workforce may be limited by its own skills needs. There is no research available to indicate what the implications are for the delivery of adult numeracy skills training in the workplace and for building the skills capacity of workers to effectively and adequately meet business needs. This study begins to redress this gap by examining the capacity of the VET workforce to address workplace numeracy skills needs, particularly in the process manufacturing industries, industries that rely greatly on the numeracy skills of its semi-skilled workers. While this study is small, it does raise questions about the capacity of the current VET workforce to address the numeracy skills gaps of existing workers. The research questions examined were as follows. The term ‘VET practitioner’ is used to refer to language, literacy and numeracy (LLN) specialists and vocational specialists collectively. What numeracy teaching qualifications and experience do VET practitioners have? To what extent do VET practitioners understand the importance of the numeracy skills of working-aged Australians generally and, specifically, of existing workers in the process manufacturing industries? What are the perceived and actual numeracy skills levels of VET practitioners? What numeracy skills gaps are preventing VET practitioners from effectively addressing the numeracy skills needs of existing workers in the process manufacturing industries? What is the capacity of Australian VET practitioners to address the numeracy skills needs of working-aged Australians? Both qualitative (self-assessments, focus group discussions, interviews) and quantitative (numeracy assessments) approaches were used. The numeracy assessments comprised an assessment tool developed specifically for the research project. The assessment questions were contextualised to the process manufacturing industries and mapped to the Australian Core Skills Framework (ACSF), with a focus on numeracy levels 3 and 41. All data-collection tools are included in the support document. The sample included 44 VET practitioners from Melbourne and Sydney who deliver workplace-based training. Of these, 20 self-identified as LLN specialists and 24 self-identified as vocational specialists. Participants were approached on the basis of their attendance at state-based Australian WELL2 Practitioner Network meetings or as vocational trainers working at registered training organisations active in the delivery of process manufacturing qualifications. Minor differences in skills, qualifications and experiences relevant to adult numeracy training were found between the two groups. The research found that participants tended to have a limited understanding of the importance of numeracy in general but demonstrated an interest and a willingness to reflect on it and adapt their thinking. It was further found that participants had a limited focus on workplace numeracy. This was confirmed by participant accounts of their experience in delivering workplace numeracy skills. The numeracy skills delivery that was identified was described by participants as at a ‘basic’ level. Most LLN specialists reported rare and only incidental delivery of workplace numeracy skills training, while most vocational specialists reported delivering workplace numeracy skills training more often, as specified in the unit requirements within qualifications. None of the participants had a specialist adult numeracy training qualification, not surprising given that only one qualification, the Graduate Certificate in Adult Numeracy Teaching, was found to be available. Six participants were identified as having an adult training specialisation that included a numeracy component, including one participant with the Vocational Graduate Certificate in Adult Language, Literacy and Numeracy Practice, one with the Advanced Diploma of Language, Literacy and Numeracy Practice in VET and four with an adult basic education qualification. Adult numeracy specialist qualifications are discussed in relation to the United Kingdom’s Skills for Life Program, where there is a separate diploma-level qualification for each specialty area, and VET practitioners seeking to qualify as an adult numeracy specialist in the United Kingdom must undertake a numeracy proficiency entry test. The research draws attention to the unreliability of numeracy self-assessment and consequently the importance of the numeracy testing of trainers, with participants generally overestimating their numeracy skills. Best practice numeracy assessment scoring methods used for this research is questioned with respect to its suitability for determining VET practitioner preparedness in the workplace context. The analysis of the numeracy assessment data showed that most participants had numeracy proficiency skills levels below the current benchmark in the Vocational Graduate Certificate in Adult Language, Literacy and Numeracy Practice, the nationally recognised qualification applicable to those with responsibility for adult literacy and numeracy training. The benchmark is questioned by the researchers as being too low by comparison with the standard suggested by international research and adopted by the United Kingdom’s Skills for Life Program. The report also explores the differences between numeracy and mathematics, the characteristics of numeracy in the workplace context and the implications for pedagogy and numeracy assessment. Based on the findings, it appears there is a mismatch between what is required to address numeracy skills needs in the process manufacturing industries and the current capacity of VET practitioners, in terms of their understanding of numeracy requirements, and their qualifications, skills and experience. 1 The Australian Core Skills Framework describes performance in five core skills: learning, reading, writing, oral communication and numeracy. Within each core skill there are five performance levels ranging from 1 (low level) through to 5 (high level performance). 2 Workplace English Language and Literacy Program