Robust automated test assembly for testlet-based tests: an illustration with the analytical reasoning section of the LSAT

In many high-stakes tests, subsets of questions (i.e., items) grouped around a common stimulus are often utilized to increase testing efficiency. These subsets of items are commonly called testlets. Since responses to items belonging to the same testlet not only depend on the test taker’s ability, but also on the correct reading, understanding, and interpretation of the stimulus, the assumption that the responses to these items are independent of one another does not always hold.\ud A mathematical model called item response theory is often applied in automated test assembly (ATA) with testlets. Testlet response theory (TRT) models have been developed to deal with dependency among items within a testlet. This report addresses some of the questions that arise in the application of TRT models to ATA. Specifically, a robust ATA method is applied. The results obtained by this method, as well as the advantages it offers, are discussed. Finally, recommendations about the use of the new method are given

Interplay of the drivers and deterrents of leisure counterfeit purchase intentions

Drawing on complementary theoretical perspectives, in this study we empirically investigate the drivers and deterrents underlying the formation of attitudes to leisure counterfeits and purchase intentions, and examine rationalization as a moderator of the attitude-intention relationship. The research model is examined via mail survey data of adult consumers using structural equations modelling. The findings indicate that perceived benefits and moral intensity significantly influence attitudes, while the impact of performance risk is negligible. The attitude and moral intensity significantly predict the purchase intention. Our analyses confirm rationalization as a salient factor moderating the relationship between attitude and behavioral intention. Based on this pattern of results, we discuss study implications in three areas: the perception of positive consequences for oneself, the perception of consequences for others, and the use of previously rarely examined consumer justifications. The finding that performance risk is not significant in affecting consumer attitude calls into question various appeals featuring adverse effects of buying counterfeits for an individual. First published online: 29 Dec 201

Applications of robust optimization to automated test assembly

Executive Summary In automated test assembly (ATA), 0-1 linear programming (0-1 LP) methods are applied to select questions (items) from an item bank to assemble an optimal test. The objective in this 0-1 LP optimization problem is to assemble a test that measures, in as precise a way as possible, the ability of candidates. Item response theory (IRT) is commonly applied to model the relationship between the responses of candidates and their ability level. Parameters that describe the characteristics of each item, such as difficulty level and the extent to which an item differentiates between more and less able test takers (discrimination) are estimated in the application of the IRT model. Unfortunately, since all parameters in IRT models have to be estimated, they do have a level of uncertainty to them. Some of the other parameters in the test assembly model, such as average response times, have been estimated with uncertainty as well. General 0-1 LP methods do not take this uncertainty into account, and overestimate the predicted level of measurement precision. In this paper, alternative robust optimization methods are applied. It is demonstrated how the Bertsimas and Sim method can be applied to take this uncertainty into account in ATA. The impact of applying this method is illustrated in two numerical examples. Implications are discussed, and some directions for future research are presented

A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales

In the last few decades, the measurement of marketing constructs has improved tremendously. Our discipline has also started to systematically catalogue our measurement knowledge in influential handbooks of marketing scales. However, at least two important issues remain. First, existing scales are often too long for administration in nonstudent samples or in applied studies. Second, existing (U.S.-developed) scales may contain items that are not informative about the underlying construct in particular countries, whereas relevant items tapping into local cultural expressions of the construct in question may be missing. To address these issues, we propose a new model that yields country-specific yet fully cross-nationally comparable short forms of unidimensional marketing scales. The procedure is based on hierarchical item response theory and optimal test design. The procedure is flexible in the sense that the researcher can specify various constraints on item content, scale length, and measurement precision. Because our procedure allows inclusion of country-specific (or “emic”) items in standardized (or “etic”) scales, it presents an important step toward resolving the emic-etic dilemma that has plagued international marketing research for decades.measurement, marketing research, marketing surveys, international marketing, scale construction, measurement invariance

Learning from subsistence marketplaces and beyond: A cross-sectoral comparison in Benin

Subsistence Marketplaces publishes studies that study consumers, entrepreneurs, and marketplaces from a broad range of low income contexts in their own right, transcending the boundaries between sectors and disciplines to derive practically meaningful implications. This paper contributes to accomplishing the mission of the new journal by contextualizing market learning theory to subsistence marketplaces. More specifically, the study links two modes of market learning (exploration and exploitation) with producers’ livelihood performance. It hypothesizes that within subsistence marketplaces, producers benefit the most from exploration but to seize a market opportunity beyond the subsistence context, BoP producers should strengthen their exploitative learning processes. We use producer survey data from 389 producers from shrimp fishing and shea butter production, two sectors selected by the Beninese Government for their development potential. The sectors vary systematically in their level of high-income market integration. We employ item-response theory to develop measures that allow to formally compare the two sectors while paying respect to their contextual differences. The results show partial support and imply that development policies and corporate procurement should support BoP producers in strengthening the appropriate learning processes

Cultural Aspects of Compulsive Buying in Emerging and Developed Economies: A cross cultural study in compulsive buying

Although several studies focused on understanding of compulsive buying in developed countries, this phenomenon remains understudied in other parts of the world. This is rather surprising since there is an increasing interest in understanding shopping behavior of consumers in emergent markets due to the growing importance of these markets. The main reason for the limited attention to compulsive buying in emerging countries is the lack of cross-culturally validated scales. In response to these calls, this paper tests measurement invariance of two prominent compulsive buying scales—the Compulsive Buying Scale (CBS) and the Compulsive Buying Index (CBI) in Western (Spain and the Netherlands) and emerging (Russia and Turkey) economies. In case of lack of invariance the reasons in terms of socio-cultural factors and country conditions are explained. The results establish the partial measurement invariance of the CBI but not the CBS. So, to study the antecedents and consequences of compulsive buying in cross-cultural contexts, the CBI is sufficient. The varying credit card ownership and usage, and different gender roles of women across countries appear to be the main reasons for lack of measurement invariance of the CBS. The percentages of compulsive buyers in emerging countries are lower than those in developed countries

Robust Automated test Assembly for testlet-Based tests:An illustration with analytical reasoning items

In many high-stakes testing programs, testlets are used to increase efficiency. Since responses to items belonging to the same testlet not only depend on the latent ability but also on correct reading, understanding, and interpretation of the stimulus, the assumption of local independence does not hold. Testlet response theory (TRT) models have been developed to deal with this dependency. For both logit and probit testlet models, a random testlet effect is added to the standard logit and probit item response theory (IRT) models. Even though this testlet effect might make the IRT models more realistic, application of these models in practice leads to new questions, for example, in automated test assembly (ATA). In many test assembly models, goals have been formulated for the amount of information the test should provide about the candidates. The amount of Fisher Information is often maximized or it has to meet a prespecified target. Since TRT models have a random testlet effect, Fisher Information contains a random effect as well. The question arises as to how this random effect in ATA should be dealt with. A method based on robust optimization techniques for dealing with uncertainty in test assembly due to random testlet effects is presented. The method is applied in the context of a high-stakes testing program, and the impact of this robust test assembly method is studied. Results are discussed, advantages of the use of robust test assembly are mentioned, and recommendations about the use of the new method are given