An argument-based validation study of the English Placement Test (EPT) – Focusing on the inferences of extrapolation and ramification

English placement tests have been widely used in higher education as post-admission assessment instruments to measure admitted English as a second language (ESL) students’ English proficiency or readiness in academic English, usually upon their arrival at universities in English-speaking countries. Unlike commercial standardized English proficiency tests, many English placement tests are locally developed with comparatively limited resources and are relatively under-investigated in the field of language testing. Even less attention has been directed to the score interpretation and the impact of placement decisions on ESL students’ English learning and academic achievement. Undoubtedly, this scarcity of research on English placement tests is inappropriate in view of their status as one of the most frequently used language testing instruments, which may exert immediate and strong impact on ESL students’ learning in general. By employing a mixed-methods approach, this dissertation project investigates the validity of test score interpretation and use of the English Placement Test (EPT) used at Iowa State University (ISU) under an argument-based validity framework. More specifically, this study started with an interpretation and use argument for the EPT, which states the score meaning and intended impact of the EPT explicitly, and focused on the last two inferences in the interpretation and use argument, namely extrapolation and ramification. The extrapolation inference links expected scores of the EPT (scores that exhibit adequate test reliability) to target scores or actual performance in the target domain. In this study, the extrapolation inference requires investigation of the relationship between ESL students’ English placement test performance and two external criteria of English performance, including the TOEFL iBT and a self-assessment. The ramification inference links the use of the EPT results to its actual impact and in this study the ramification inference requires investigation of the impact of the placement decisions in a specific educational context. For the extrapolation inference, quantitative data such as test performance data on the EPT, the TOEFL iBT, and the self-assessment were collected and analyzed using multitrait-multimethod (MTMM) analysis techniques. The findings indicated that the EPT was found to have moderate relationships with the TOEFL iBT and weak to moderate relationships with the self-assessment. The EPT showed some of the expected convergent correlations as well as discriminant correlations based on the MTMM correlation coefficient matrix as well as the factor loading parameters in a correlated trait-correlated uniqueness (CTCU) model. For the ramification inference, three types of analyses were conducted to seek support with regard to 1) test stakeholders’ perceptions of the EPT placement decisions, 2) the impact of the EPT placement on ESL students’ English learning, and 3) the relationship between ESL students’ EPT performance and their first-semester academic achievement. The interviews with test stakeholders were coded and analyzed to identify statements indicating their perceptions of the impact of the placement decisions. The qualitative findings are also utilized to help interpret the quantitative findings. Multiple paired-samples t-tests are used to investigate ESL students’ progress in the ESL courses that they were placed into. In addition, a structural equation modeling (SEM) approach was used to model the relationship among students’ performance on the EPT, ESL courses, and their first-semester GPA, mediated by individual difference constructs, such as learning motivation, academic self-efficacy, and self-regulated learning strategies. The qualitative analyses of the interviews with four groups of test stakeholders show that the interviewed ESL students in general experienced initial frustration regarding the placement decisions, in retrospect, they understood why they were placed into ESL courses and appreciated the benefits of taking the required courses, especially ESL writing courses. The ESL course instructors were satisfied with the placement accuracy, even though occasionally they identified a few cases of potentially misplaced students in the ESL courses. The interviewed undergraduate academic advisors showed positive perceptions of the EPT and the placement decisions. They also reported observing that the majority of the ESL advisees were receptive to the EPT placement decisions. The analyses of ESL course performance data collected at the beginning and the end of the course indicate that ESL students in Engl99L, an ESL listening course focusing on listening strategies, made statistically significant progress in terms of score gain on the same listening test administered at two time points. However, only nine out of 38 ESL students made satisfactory progress with reference to the course standard. Students in Engl101B (a lower-level ESL academic English writing course) and Engl101C (a higher-level ESL academic English writing course) did not show much progress in terms of lexical complexity, syntactic complexity, and grammatical accuracy. However, the Engl101C students on average wrote longer essays at the end of the course. Based on the ratings of the essays written in the final exams using the EPT scoring rubric, 14 out 18 Engl101B students (77.8%) and eight out of 16 Engl101C students (50%) showed satisfactory progress in these classes and were deemed ready for the next level of English study. The SEM analysis results indicate that ESL students’ EPT performance had significant and direct impact on their academic achievement. What’s more, students’ EPT performance predicted their academic self-efficacy and affected extrinsic goal orientation. However, these motivational factors did not have direct impact on academic achievement. The findings in this study contribute to building the validity argument for the EPT with two of the assumptions underlying the warrant for the extrapolation inference and ramification inference found supported and the other three partially supported. This findings in this study contributed to a better understanding of the score interpretation and use of the EPT at Iowa State University through constructing a validity argument. These findings shed light on the future development of the EPT and other similar English placement tests. The findings in this study as well as the research methodology can be informative for other institutions where English placement tests are used

A survey of statistical network models

Networks are ubiquitous in science and have become a focal point for discussion in everyday life. Formal statistical models for the analysis of network data have emerged as a major topic of interest in diverse areas of study, and most of these involve a form of graphical representation. Probability models on graphs date back to 1959. Along with empirical studies in social psychology and sociology from the 1960s, these early works generated an active network community and a substantial literature in the 1970s. This effort moved into the statistical literature in the late 1970s and 1980s, and the past decade has seen a burgeoning network literature in statistical physics and computer science. The growth of the World Wide Web and the emergence of online networking communities such as Facebook, MySpace, and LinkedIn, and a host of more specialized professional network communities has intensified interest in the study of networks and network data. Our goal in this review is to provide the reader with an entry point to this burgeoning literature. We begin with an overview of the historical development of statistical network modeling and then we introduce a number of examples that have been studied in the network literature. Our subsequent discussion focuses on a number of prominent static and dynamic network models and their interconnections. We emphasize formal model descriptions, and pay special attention to the interpretation of parameters and their estimation. We end with a description of some open problems and challenges for machine learning and statistics.Comment: 96 pages, 14 figures, 333 reference

Theoretical and Practical Advances in Computer-based Educational Measurement

This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology

INVALSI data: assessments on teaching and methodologies

The school system has always aimed to achieve quality teaching, which is able, on the one hand, to give adequate responses to the expectations of all the stakeholders and, on the other, to introduce tools, actions, and checks through which the training offer can be constantly improved. This process is undoubtedly linked to scientific research. Researchers and Academics start from the data available to them or collect new ones, to discover and/or interpret facts and to find answers and new cues of reflection. A favorable environment for this work was the Seminar “INVALSI data: a research and educational teaching tool”, in its fourth edition in November 2019. The volume consists of six chapters, which are arise within the aforementioned Seminar context and, while dealing with heterogeneous topics, offer important examples of research both on teaching and on the methodologies applied to it. As a Statistical Service, which for years has taken care of the collection and dissemination of data, we hope that in this, as in the other volumes of the series, the reader will find confirmation of the importance that data play, both in scientific research and in practice in classroom

INVALSI data: assessments on teaching and methodologies

The school system has always aimed to achieve quality teaching, which is able, on the one hand, to give adequate responses to the expectations of all the stakeholders and, on the other, to introduce tools, actions, and checks through which the training offer can be constantly improved. This process is undoubtedly linked to scientific research. Researchers and Academics start from the data available to them or collect new ones, to discover and/or interpret facts and to find answers and new cues of reflection. A favorable environment for this work was the Seminar “INVALSI data: a research and educational teaching tool”, in its fourth edition in November 2019. The volume consists of six chapters, which are arise within the aforementioned Seminar context and, while dealing with heterogeneous topics, offer important examples of research both on teaching and on the methodologies applied to it. As a Statistical Service, which for years has taken care of the collection and dissemination of data, we hope that in this, as in the other volumes of the series, the reader will find confirmation of the importance that data play, both in scientific research and in practice in classroom

The participation of older people with and without dementia in public space, through the lens of everyday technology use

Participation in activities and places within public space has been linked to numerous health benefits and yet, little is known about participation among older people with and without dementia. Insights about participation in activities and places within public space can contribute to the somewhat ambiguous definition of participation, as “involvement in a life situation”, by acknowledging the complexity and interrelatedness of subjective, social, contextual, temporal, and technological aspects of participation. Thus, the overarching aim of the four studies was to explore participation in activities and places within public space, among older people with and without dementia in two European countries (Sweden and UK), and to evaluate how different aspects, such as the relevance and perceived ability to use Everyday Technologies (ETs), interact with and influence participation, over time. Across all studies, interviews used the Participation in Activities and Places Outside Home Questionnaire (ACT-OUT) and the Everyday Technology Use Questionnaire (ETUQ), in order to focus on the perspectives of older people with and without dementia themselves. Study one explored stability and changes in participation in places visited within public space in relation to the relevance of ETs used in public space, among a baseline Swedish sample. Study two utilised ordinal regression to investigate the ways in which perceived risks and ET use were associated with out-of-home participation, among a UK sample of older people. Using a mixed methods design and data visualisations, study three delved into aspects of social participation in more depth, including ET use and social deprivation of the living environment, among two UK sub-samples of older people with and without dementia. Study four’s longitudinal design and multilevel modelling deepened the knowledge about how use of ET outside home, relates to participation in places visited within public space among a Swedish sample of older people with dementia over time. Study one’s findings demonstrated a statistically significant positive association between a higher person measure of ability to use ETs and higher participation in places visited within public space, among the Swedish sub-sample of older people with dementia but not those without dementia. According to the ordinal regression model in study two, a higher probability of ET use was associated with a higher level of out-of-home participation, among the UK sample of older people. By elucidating motivators, considerations that require extra attention, and management strategies among UK sub-samples of older people with and without dementia, study three provided insights into the nuances of social participation. Finally, study four’s findings revealed that decreasing use of ET outside home was associated with decreasing participation in places visited within public space, in a statistically significant way when accounting for age. In summary, this thesis contributes empirical insights about the participation of older people with and without dementia in activities and places within public space, through the lens of ET use. Such knowledge can be used to develop targeted health and social care planning and the design of more inclusive places, technologies, and services

Statistical Modelling

The book collects the proceedings of the 19th International Workshop on Statistical Modelling held in Florence on July 2004. Statistical modelling is an important cornerstone in many scientific disciplines, and the workshop has provided a rich environment for cross-fertilization of ideas from different disciplines. It consists in four invited lectures, 48 contributed papers and 47 posters. The contributions are arranged in sessions: Statistical Modelling; Statistical Modelling in Genomics; Semi-parametric Regression Models; Generalized Linear Mixed Models; Correlated Data Modelling; Missing Data, Measurement of Error and Survival Analysis; Spatial Data Modelling and Time Series and Econometrics

A Group Contingency Plus Self-Management Intervention Targeting At-Risk Secondary Students' Class-work and Active Responding

Incomplete written work and lack of active classroom responding are reported to be obstacles to secondary students' learning. Effective interventions found in meta-analytic reviews of the current research literature include: differential reinforcement of desired behaviors through group contingencies and self-management strategies (Hoagwood et al., 2007; Prout & Prout, 198; Stage & Quiroz, 1997). However, the current studies have mostly focused on elementary school settings. The purpose of the present study is to show that an independent group contingency combined with self-management strategies and randomized-reinforcer components can increase the amount of written work and active classroom responding in high school students. Three remedial reading classes with a total of 15 students participated in this study. Students used self-management strategies during independent reading time to increase the amount of writing in their reading logs. They used self-monitoring strategies to record whether or not they performed expected behaviors in class. A token economy was used to provide positive reinforcement for target responses. The results were analyzed through visual inspection of graphs and effect size computations and showed that the intervention increased the total amount of written words in the students' reading logs and overall classroom academic responding