Using O*NET to Develop a Framework of Job Characteristics to Potentially Improve the Predictive Validity of Personality Measures

It has long been theorized that we can improve prediction of job-related behavior from measures of personality by identifying job characteristics that allow for the expression of individual differences (e.g., Mischel, 1968). Using O*NET data, the current paper develops a framework for job characteristics that could improve the extent to which we can predict behavior from personality. More specifically, it investigates relationships between Work Styles, Generalized Work Activities, and Work Context variables. Job characteristics varied in importance as a function of four Work Styles composites: achievement, people orientation, stability, and attention to detail, and the relationships were largely consistent with the tenants of Trait Activation Theory (Tett & Burnett, 2003). In addition to limitations and future directions, the discussion section contains implications of the current study for practitioners, including implications for hiring practices and job placement

Facets of conscientiousness and their differential relationships with cognitive ability factors

This study examined relationships between conscientiousness facets and both broad factors of cognitive ability and collegiate GPA. Students responded to 117 Conscientiousness items and 15 cognitive tests demarcating fluid intelligence, crystallized intelligence, quantitative reasoning, visual processing, and broad retrieval ability. Confirmatory factor analysis replicated the eight-factor model found in MacCann, Duckworth, and Roberts (2009). Conscientiousness facet correlations with Cognitive Ability and GPA revealed that Cautiousness exhibited the highest correlation with Cognitive Ability, while Industriousness showed the strongest relationship with GPA. Procrastination Refrainment was the only facet negatively related to Cognitive Ability. Implications of these results are discussed in light of previ- ous research and the potentially moderating effect of high- versus low-stakes testing on the relationship between conscientiousness and cognitive ability

Egocentrism in Perceptions of Distributive Justice: When Favorable Outcomes Are Unfair Outcomes

71 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2006.Previous research has suggested that judgments of distributive justice are egoistic (e.g. Greenwald, 1983). That is, people tend to think that outcomes that favor themselves are fair outcomes. However, I propose that judgments of distributive justice are often a product of egocentrism (e.g. Kruger, 1999). That is, people tend to focus more on their own contribution than on the contribution of their collaborators when making distributive justice judgments. This leads to a prediction of an interaction of amount of contribution with favorability of outcome, such that people who have contributed a lot to a task will think it is fairer for them to receive more favorable outcomes than others than to receive less favorable outcomes than others. However, this effect should be attenuated, or even reversed, when participants have contributed little to the task. Importantly, this should be true even when their collaborators have contributed just as much or just as little as they have. In the primary study, two participants answered trivia questions as part of a "quiz bowl team", were paid money for their performance, and were asked to evaluate the distributive justice of the payments. Results supported the hypothesis. Thus, favorable outcomes may be seen as fair when contributions are high, but unfair when contributions are low.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD

Situational Judgment Tests as a method for measuring personality: Development and validity evidence for a test of Dependability

Gefördert durch den Publikationsfonds der Universität Kasse

A Big Five-Based Multimethod Social and Emotional Skills Assessment: The Mosaic™ by ACT® Social Emotional Learning Assessment

A focus on implementing social and emotional (SE) learning into curricula continues to gain popularity in K-12 educational contexts at the policy and practitioner levels. As it continues to be elevated in educational discourse, it becomes increasingly clear that it is important to have reliable, validated measures of students’ SE skills. Here we argue that framework and design are additional important considerations for the development and selection of SE skill assessments. We report the reliability and validity evidence for The Mosaic™ by ACT® Social Emotional Learning Assessment, an assessment designed to measure SE skills in middle and high school students that makes use of a research-based framework (the Big Five) and a multi-method approach (three item types including Likert, forced choice, and situational judgment tests). Here, we provide the results from data collected from more than 33,000 students who completed the assessment and for whom we have data on various outcome measures. We examined the validity evidence for the individual item types and the aggregate scores based on those three. Our findings support the contribution of multi-method assessment and an aggregate score. We discuss the ways the field can benefit from this or similarly designed assessments and discuss how the assessment results can be used by practitioners to promote programs aimed at stimulating students’ personal growth

Identifying the Most Important 21st Century Workforce Competencies: An Analysis of the Occupational Information Network (O*NET) ETS Research Report Series EIGNOR EXECUTIVE EDITOR Identifying the Most Important 21st Century Workforce Competencies: An Analy

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Consistent with this conclusion, a correlation of component scores with wages found that 4 of these 5 competencies were strongly related to wages, with the exception being teamwork

Building a Case to Develop Noncognitive Assessment Products and Services Targeting Workforce Readiness at ETS ETS Research Memorandum Series Building a Case to Develop Noncognitive Assessment Products and Services Targeting Workforce Readiness at ETS

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. ETS, the ETS logo, GRE, LISTENING. LEARNING. LEADING., and TOEIC are registered trademarks of Educational Testing Service (ETS). iSKILLS and PRAXIS are trademarks of ETS. SAT is a registered trademark of the College Board. As part of its nonprofit mission, ETS conducts and disseminates the results of research to advance quality and equity in education and assessment for the benefit of ETS's constituents and the field. To obtain a PDF or a print copy of a report, please visit: http://www.ets.org/research/contact.html i Abstract The goal of this paper is to establish the case for conducting research on the readiness of individuals for the workforce as part of the Workforce Readiness Initiative at ETS, with specific emphasis, at least initially, on noncognitive indicators of readiness. We begin by defining a conceptual framework that encompasses noncognitive constructs and measures, followed by a brief review of the literature highlighting the importance of noncognitive predictors in education and the workforce. Next, we examine the importance of research on workforce readiness and consider how ETS can conduct workforce readiness research. Finally, we give an overview of work accomplished in this area at ETS, include an action plan of workforce research in progress, and summarize future planned directions. Of the five main personality factors, conscientiousness has been shown to be the most Given the large body of evidence supporting the importance of noncognitive variables in education and in the workforce, there are a number of ways in which researching noncognitive predictors of workforce skills serves ETS as an organization. This paper highlights arguments from a policy perspective, as well as arguments based on scientific merit and business practices. Importance of Workforce Readiness Research ETS has emerged as a leader in alerting the American public on the need for a better prepared workforce. The ETS Policy Information Report, America's Perfect Storm The First Force The Second Force Research on the noncognitive predictors of workplace performance clearly addresses issues related to ongoing structural changes to the economy. Labor markets are different than in decades past, due in part to "industrial and corporate restructuring, declines in unionization, technological change, and globalization" (Kirsch et al., 2007, p. 6). These changes favor workers who possess a different set of skills than were required under the old economic structure, and several of the most important of these skills can be characterized as noncognitive. Interestingly, these noncognitive skills were rated more important than skills traditionally taught 5 and assessed by high schools and colleges. In short, the business community is explicitly stating that classic cognitive skills are not enough for workplace success and that noncognitive skills are important as well. Addressing these business needs is a vital component of pursuing workforce research at ETS. The Third Force The third force identified by America's Perfect Storm Specifically, there will be a dramatic increase in racial and ethnic diversity over this time period. The changing makeup of the American populace will result in an increase in the importance of noncognitive workforce assessments. Although cognitive assessments consistently demonstrate 6 validity in workforce studies (and are legally defensible when shown to be directly relevant to the job), the use of traditional cognitive assessments alone can present problems in business contexts. For example, using traditional cognitive instruments as the sole predictor in selection contexts typically leads to some racial and ethnic minorities being selected at a lower rate than whites. However, as previously reviewed, noncognitive assessments typically result in less disparate impact than do traditional cognitive assessments. Thus, an increase in noncognitive workforce assessment has the potential to result in selection and training practices that result in a more diverse workplace for the 21st century workforce. Developing the Case for ETS Involvement in Workforce Readiness Research The authors of America's Perfect Storm Mission Consistency Any work conducted by ETS should be consistent with its mission and its status as a taxexempt nonprofit organization. The first part of the ETS mission is "to advance quality and equity in education by providing fair and valid assessments, research and related services." The lobbying restriction. This means that the organization cannot participate in political activities. This will not change as a result of conducting workforce-readiness research. The public benefit test. "The organization must operate for the advantage of public, rather than private, interests. Private interests can be benefited, but only incidentally. Further, the principal beneficiaries of the organization's activities must be sufficiently numerous and well-defined so that the community is, in some way, served" (Bennett, Research on workforce readiness is also consistent with the history of ETS. For example, Carl Brigham, inventor of the SAT, was interested in using assessment as a guide for instruction in addition to using it for selection Having outlined the policy and mission-related goals in conducting workforce research, how can ETS contribute to understanding workforce readiness? To address these policy goals, ETS can develop valid, reliable assessments of noncognitive skills and identify workers who have not accumulated these skills from their previous education. Next, ETS can develop interventions to improve these skills and follow up by assessing the effectiveness of these interventions. Examples of potential avenues of research from a policy perspective are underscored in the section on future directions below. 9 Scientific Merit From a scientific perspective, ETS can provide unique contributions to the understanding of the relation between noncognitive constructs and performance. With access to large samples of graduating students, ETS can leverage its connections by following students after graduation to monitor workforce performance. Workforce research can contribute towards a better solution for the longstanding criterion problem, one of the most important and difficult problems in workforce and general validation research Plan section outlining the current Call Center Study at ETS. In addition, ETS has several advantages over typical firms that study workforce readiness. First, ETS's status as a nonprofit organization places it in a unique position to make substantial scientific contributions. That is, because it is not accountable to shareholders, ETS is free to take risks and attempt to answer more difficult workforce research questions than can the average consulting firm more concerned with its financial bottom line. Furthermore, ETS's status as a world-class testing agency provides it with resources that would not be available to most firms. For example, ETS's psychometric capabilities are unmatched by most, if not all, for profit consulting firms. Business Value Bennett (2008) stated that, in order to fund its mission, a nonprofit organization such as ETS must be responsive to market needs. As evidenced by Casner-Lotto and Barrington It is an open issue whether noncognitive skill assessments at ETS ought to be designed or advertised for purposes of selection or hiring decisions. But selection is only one of many possible uses for workforce assessments. ETS can provide unique insight for assessments 10 designed for training and development, promotion and succession planning purposes. These kinds of noncognitive assessments can help organizations understand where employees might best be placed to improve skills or how to target interventions to promote the skills they would like to improve. Much as the Myers Briggs Type Indicator (MBTI) is widely used to assess employee skills and fit in an organization, ETS has the capability to develop and test products that would identify skilled workers for an organization. The MBTI purports to measure 16 personality types along four dimensions (extraversion-introversion, sensing-intuition, thinking-feeling, and judgment-perception). Despite suffering from significant psychometric problems in terms of both validity and reliability (see The TOEIC ® test is an example of a successful ETS product that assesses skills and learning in a business context, as evidenced by its approximately 6.6 million annual tests administered. By measuring speaking, writing, listening, and reading proficiency in workplace English, the TOEIC not only assesses learning in a workplace context, but also meets a market need from both the business and educational communities. The PRAXIS™ series of teacher licensure and certification assessments also represents an existing workforce assessment at ETS that focuses on the profession of teaching, identifying, and measuring general and subjectspecific teaching skills. Finally, ETS has produced an iSkills™ certification assessment that incorporates computer literacy and critical thinking skills, an assessment designed to hone the proficiencies of college students as they prepare for the workforce. Just as each of these products was developed at ETS to assess workforce needs, creating products that assess important noncognitive skills can contribute to market needs for noncognitive measures that may lead to improved workforce readiness and success. ETS should thus seek to develop valid and reliable assessments of constructs such as teamwork and work ethic (conscientiousness) that have been previously rated as very important to workforce success Summary of Perspectives In the end, the goal of workforce research in an educational testing organization i