Predictors of Adolescents’ Interest in Stem Majors and Careers

Advanced Research Winner 2019: The United States currently faces a shortage of qualified workers in fields related to science, technology, engineering, and math (STEM). The first critical step in preventing the labor shortage in STEM is understanding the factors that guide adolescents toward STEM pursuits. Drawing on Eccles’ expectancy-value theory (EVT), the current study aims to identify factors that are relevant to adolescents’ interest in STEM majors and careers. Data were collected from 629 adolescents (Mage = 16.09). Participants attended a high school in northern California and predominantly identified as Asian American (82% of the sample). Preliminary analyses revealed that adolescent boys had higher STEM self-expectancies than did adolescent girls, whereas there was no gender difference in STEM values. Consistent with expectations, multiple regression demonstrated that STEM self-expectancies and values accounted for a significant amount of variance in participants’ interest in STEM majors and careers. STEM value was an especially strong predictor; adolescents tended to be most interested in STEM pursuits when they were also high in STEM value. Moderation analyses showed that the association between STEM value and interest in STEM majors and careers was stronger for girls than for boys. As a whole, this study’s findings suggest that valuing and enjoying STEM pursuits during high school could be an important antecedent of pursuing a STEM major and a STEM career later in life

Strengthening Incentives for Student Effort and Learning: Michigan’s Merit Award Program?

[Excerpt] One of the primary reasons American students learn a good deal less during secondary school than students in other industrialized nations is that they devote less time and intellectual energy to the task.1 Accountability systems designed to get teachers to try harder and set higher standards will not produce more student learning if [as one high school teacher put it] “students are sitting back in their desks, arms crossed, waiting for their teachers to make them smart (Zoch, 1998, p. 70).” Learning is not a passive act; it requires the time and active involvement of the learner. In a classroom with 1 teacher and 25 students, there are 25 learning hours spent for every hour of teaching time. Learning takes work and that work is generally not going to be as much fun as hanging out with friends or watching TV. If students cannot be motivated to give up some time socializing or watching TV so that they can learn difficult material and develop high level skills, the time and talents of teachers will be wasted

COMPUTER ADAPTIVE TESTING (CAT) DESIGN; TESTING ALGORITHM AND ADMINISTRATION MODE INVESTIGATION

Since the advent of technology to transform education, the use of computer technology has pervaded many areas of fields of study such as language learning and testing. Chapelle (2010) distinguishes three main motives for using technology in language testing: efficiency, equivalence and innovation. Computer as a frequently used technological tool has been widely inspected in the field of language assessment and testing. Compute-adaptive language test (CALT) is a subtype and subtest of computer-assisted language test because it is administered at computer terminal or on personal computer. The issue that currently needs more attention and prompt investigation of researchers is to study the testing mode and paradigm effects on comparability and equivalency of the data obtained from two modes of presentation, i.e. traditional paper-and-pencil (PPT) and computerized tests. To establish comparability and equivalency of computerized test with its paper-and-pencil counterpart is of importance and critical. Then, in this study, the researcher indicate that in order to replace computer-adaptive test with conventional paper-and-pencil one, we need to prove that these two versions of test are comparable, in other words the validity and reliability of computerized counterpart are not violated.  Article visualizations

Testi täitmise motivatsioon madala ja kõrge olulisusega testimise kontekstides

Väitekirja elektrooniline versioon ei sisalda publikatsiooneHaridus- ja psühholoogiavaldkonnas kasutatakse teste inimeste teadmiste ja oskuste hindamiseks. Tihti ei ole selliste testide tulemused testitavate endi jaoks tähenduslikud, kuigi võivad olla olulised mõnel teisel tasandil – näiteks võidakse nende alusel võrrelda omavahel õpilasi või õpilaste gruppe koolide ja riikide lõikes. Varasemad uuringud on näidanud, et keskmiselt on testitäitjate jaoks madala olulisega testide tulemused madalamad kui kõrge olulisusega testide puhul. Ka valiidsuse teoreetilised käsitlused viitavad sellele, et testi tulemuste tõlgendamisel tuleks arvestada erinevate teguritega, sh motivatsiooniga, mis võivad tulemust mõjutada, aga ei ole osa konstruktist, mida soovitakse hinnata. Testi täitmise motivatsiooni (TTM) käsitletakse peamiselt ootuse-väärtuse teooria raamistikus. Selle alusel on motivatsiooni tekkimine seotud erinevate teguritega, näiteks inimese eelnevad kogemused, väärtused ja uskumused. Seega TTM ei ole seotud vaid testi ja testimise olukorraga vaid ka testitava enda sisemiste protsessidega. Sellest lähtuvalt võib eeldada, et TTM võib olla tulemustega seotud ka kõrge olulisusega testimise olukorras. Selleks, et TTM.iga oleks võimalik tulemuste tõlgendamise arvestada, on oluline seda hinnata. Peamiselt on hindamiseks kasutatud enesekohaseid küsimustikke, kuid kuna vastused nendele ei pruugi olla piisavalt objektiivsed, on välja pakutud ka alternatiivseid TTM-i hindamise meetodeid, näiteks ajalist pingutust vastamisel (APV, Wise ja Kong, 2005). APV alusel antakse hinnang testitäitja testile pühendatud pingutusele selle alusel, kuidas ta kasutas testi ülesannete lahendamisel selleks ettenähtud aega. Doktoritöös uuriti, kuidas avaldub testi täitmise pingutus nii madala kui ka kõrge olulisusega testimise olukordades. Pingutuse hindamiseks kasutati nii enesekohast küsimustikku kui ajalisi pingutust iseloomustavad indikaatoreid, sh APV. Leiti, et testi täitmise pingutus on testi tulemusega seotud nii madala kui ka kõrge olulisusega testis, kuigi ootuspäraselt seletab see suurema osa variatiivsusest madala olulisusega testi puhul. Veel leiti, et enesehinnatud pingutus ja testi täitmise aja põhjal arvutatud pingutuse näitaja võivad kirjeldada motivatsioonispektri erinevaid osi ning et mõlemad võivad anda väärtuslikku informatsiooni testi täitmise motivatsiooni kohta.  In the fields of education and psychology, tests are used to measure people's knowledge and skills. However, the results of such tests are often not personally relevant to the test-takers, although these may be relevant on other levels – for example, test results may be used to compare students or groups of students across schools and countries. Previous studies have shown that results on low-stakes tests, that do not have significant consequences for test-takers, are lower compared to results in high-stakes tests. Validity theory also suggests that different factors contributing to construct irrelevant variance, including motivation, should be considered when interpreting test results. As test-taking motivation (TTM) is mainly seen in the framework of expectation-value theory, it can be inferred that the motivation to complete a test can also manifest itself in a situation that is generally considered to be high-stakes. Therefore, it is important to assess TTM. Mainly self-report questionnaires have been used for this, but as answers to these may not always be sufficiently objective, alternative methods for assessing test-taking motivation have been proposed such as Response Time Effort (RTE, Wise and Kong, 2005). RTE gives an indication of the amount of effort used in the test based on test-taking times. In this doctoral dissertation, it was investigated how the effort to complete the test manifests itself in both low-stakes and high-stakes testing contexts. Test-taking effort was assessed with a self-report questionnaire and time-based measures of effort, including RTE. Test-taking effort was related to test performance in both low-stakes and high-stakes testing contexts, although test-taking effort explained a larger part of the variance in the low-stakes test. It was reasoned that self-reported effort and RTE describe different parts of the motivation spectrum, and that both can provide valuable information about test-taking motivation.https://www.ester.ee/record=b550899

Is the literacy achievement of teenage boys poorer than that of teenage girls, or do estimates of gender gaps depend on the test? A comparison of PISA and PIAAC

Data from international large-scale assessments (ILSAs) of schooled populations indicate that boys have considerably poorer literacy skills than girls. New evidence from a household-based ILSA—Organisation for Economic Cooperation and Development Survey of Adult Skills (PIAAC)—indicates that the gender gap in literacy is negligible, even though its assessment framework is similar to that of one of the most widely used school-based assessments, the Program for International Student Assessment (PISA). Individual-level data from 15-, 16-, and 17-year-old teens in countries that administered both assessments were used to estimate and compare literacy gender gaps in the two assessments, after accounting for differences in target population, response rates, scoring scheme, test length, mode of delivery, prevalence of items involving different stimuli in the two assessments (e.g., types of texts), and cognitive processes test-takers need to engage in to solve assessment items (e.g., accessing and retrieving information or reflecting and evaluating information presented in the text). These differences explain only part of the differences across the two studies in estimated literacy gender gaps: Even when these factors are considered, gender gaps remain large in PISA and small (though imprecisely estimated) in PIAAC. The potential roles of test-taking motivation and administration conditions in explaining differences across the studies and implications for research and policy are discussed

Rise to the Challenge or Not Give a Damn: Differential Performance in High vs. Low Stakes Tests

This paper studies how different demographic groups respond to incentives by comparing performance in the GRE examination in "high" and "low" stakes situations. The high stakes situation is the real GRE examination and the low stakes situation is a voluntary experimental section of the GRE that examinees were invited to take immediately after they finished the real GRE exam. We show that males exhibit a larger difference in performance between the high and low stakes examinations than females, and that Whites exhibit a larger difference in performance between the high and low stakes examinations relative to Asians, Blacks, and Hispanics. We find that the larger differential performance between high and low stakes tests among men and whites can be partially explained by the lower level of effort invested by these groups in the low stake test.gender, competition, incentives, GRE, high stakes, low stakes, test score gap

Identifying low test-taking effort during low-stakes tests with the new Test-taking Effort Short Scale (TESS) – development and psychometrics

Background: Low-stakes tests are becoming increasingly important in international assessments of educational progress, and the validity of these results is essential especially as these results are often used for benchmarking. Test scores in these tests not only mirror students’ ability but also depend on their test-taking effort. One way to obtain more valid scores from participating samples is to identify test-takers with low test-taking effort and to exclude them from further analyses. Self-assessment is a convenient and quick way of measuring test-taking effort. We present the newly developed Test-taking Effort Short Scale (TESS), which comprises three items measuring attainment value/intrinsic value, utility value, and perceived benefits, respectively. Methods: In a multicenter validation study with N = 1837 medical students sitting a low-stakes progress test we analyzed item and test statistics including construct and external validity. Results: TESS showed very good psychometric properties. We propose an approach using stanine norms to determine a cutoff value for identifying participants with low test-taking effort. Conclusion: With just three items, TESS is shorter than most established self-assessment scales; it is thus suited for administration after low-stakes progress testing. However, further studies are necessary to establish its suitability for routine usage in assessment outside progress testing

Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Integrating Timing Considerations to Improve Testing Practices

Integrating Timing Considerations to Improve Testing Practices synthesizes a wealth of theory and research on time issues in assessment into actionable advice for test development, administration, and scoring. One of the major advantages of computer-based testing is the capability to passively record test-taking metadata—including how examinees use time and how time affects testing outcomes. This has opened many questions for testing administrators. Is there a trade-off between speed and accuracy in test taking? What considerations should influence equitable decisions about extended-time accommodations? How can test administrators use timing data to balance the costs and resulting validity of tests administered at commercial testing centers? In this comprehensive volume, experts in the field discuss the impact of timing considerations, constraints, and policies on valid score interpretations; administrative accommodations, test construction, and examinees’ experiences and behaviors; and how to implement the findings into practice. These 12 chapters provide invaluable resources for testing professionals to better understand the inextricable links between effective time allocation and the purposes of high-stakes testing