La familia de falacias "enseñando para el examen"

This article explains the various meanings and ambiguities of the phrase “teaching to the test” (TttT), describes its history and use as a pejorative, and outlines the policy implications of the popular, but fallacious, belief that “high stakes” testing induces TttT which, in turn, produces “test score inflation” or artificial test score gains. The history starts with the infamous “Lake Wobegon Effect” test score scandal in the US in the 1980s. John J. Cannell, a medical doctor, discovered that all US states administering national norm-referenced tests claimed their students’ average scores exceeded the national average, a mathematical impossibility. Cannell blamed educator cheating and lax security for the test score inflation, but education insiders managed to convince many that high stakes was the cause, despite the fact that Cannell’s tests had no stakes. Elevating the high stakes causes TttT, which causes test score inflation fallacy to dogma has served to divert attention from the endemic lax security with “internally administered” tests that should have encouraged policy makers to require more external controls in test administrations. The fallacy is partly responsible for promoting the ruinous practice of test preparation drilling on test format and administering practice tests as a substitute for genuine subject matter preparation. Finally, promoters of the fallacy have encouraged the practice of “auditing” allegedly untrustworthy high-stakes test score trends with score trends from allegedly trustworthy low-stakes tests, despite an abundance of evidence that low-stakes test scores are far less reliable, largely due to student disinterestEste artículo explica los diversos significados y ambigüedades de la frase "enseñar para el examen" (TttT: teaching to the test en inglés), describe su historia y su uso como un peyorativo, y describe las implicaciones políticas de la creencia popular, pero falaz, que las pruebas de a “gran escala” inducen TttT que, a su vez, produce una "inflación en la calificación obtenida en el examen" o ganancias em cuanto a los puntos obtenidos en la prueba. La historia comienza con el infame escándalo de la puntuación de la prueba "Lake Wobegon Effect" en los Estados Unidos en los años ochenta. John J. Cannell, un médico, descubrió que todos los estados de los Estados Unidos que administraban pruebas nacionales con referencias normativas afirmaban que los puntajes promedio de sus estudiantes excedían el promedio nacional, una imposibilidad matemática. Cannell atribuyó a los educadores el engaño y la seguridad laxa por la inflación de la puntuación de los exámenes, pero los expertos en educación lograron convencer a muchos de que las pruebas a gran escala eran la causa, a pesar de que las pruebas de Cannell no tenían ninguna fiabilidad. Exagerar las pruebas a gran escala hace que TttT hace que la falla de la inflación de la puntuación de la prueba al dogma haya servido para desviar la atención de la seguridad laxa endémica con pruebas "internamente administradas" que deberían haber alentado a los responsables políticos a exigir más controles externos en las administraciones de las pruebas. La falacia es en parte responsable de promover la práctica ruinosa en la preparación de las pruebas en el formato de prueba y la administración de pruebas prácticas como un sustituto de la preparación de la materia original. Por último, los promotores de la falacia han fomentado la práctica de "auditar" tendencias de determinadas puntuación en las pruebas a gran escala con las tendencias de puntuación presuntamente confiables de las pruebas de baja exigencia, a pesar de la abundancia de pruebas donde las puntuaciones de las pruebas a menor escala son mucho menos confiables debido al desinterés de los estudiante

CEO pay and the Lake Wobegon effect

Working PaperIn this paper, we propose a new explanation for the recent increase in CEO pay at US firms. Our explanation, which is based on asymmetric information in financial markets, is motivated by a recent observation made by former DuPont CEO Edward S. Woolard, Jr.: "The main reason (CEO) compensation increases every year is that most boards want their CEO to be in the top half of the CEO peer group, because they think it makes the company look strong. So when Tom, Dick, and Harry receive compensation increases in 2002, I get one too, even if I had a bad year.... (This leads to an) upward spiral" (Elson, 2003). We present a game-theoretic model of this phenomenon, which is known in the business press as the "Lake Wobegon Effect." Our model has three key features: (i) there is asymmetric information regarding the manager's ability to create value at the firm; (ii) the pay package given to the manager must convey information about the manager's ability to create value at the firm; and, (iii) the firm must have some preference for favorably affecting outsiders' perceptions of firm value. We characterize the perfect Bayesian equilibrium of this model, identify conditions under which pay is distorted upward relative to a full-information benchmark, and then embed our model in a simple assortative matching framework. Our analysis offers a potential explanation across-country differences in CEO pay growth, and suggests that greater shareholder involvement in the pay process may be counterproductive

Hiring above the m-th best candidate: a generalization of records in permutations

The hiring problem is a simple model of on-line decision- making under uncertainty. As in many other such models, the input is a sequence of instances and a decision must be taken for each instance depending on the subsequence examined so far, while nothing is known about the future. One famous example of on-line decision-making the secretary problem, formally introduced in the early sixties. Broder et al. (2008) introduced the hiring problem as an extension of the secretary problem. Instead of selecting only one candidate, we are looking for selecting (hiring) many candidates to grow up a small company. In this context, a hiring strategy should meet two demands: to hire candidates at some reasonable rate and to improve the average quality of the hired staff. Soon afterwards, Archibald and Martinez (2009) introduced a discrete model of the hiring problem where candidates seen so far could be ranked from best to worst without the need to know their absolute quality scores. Hence the sequence of candidates could be modeled as a random permutation. Two general families of hiring strategies were introduced: hiring above the m-th best candidate and hiring in the top P % quantile (for instance, P = 50 is hiring above the median). In this paper we consider only hiring above the m-th best candidate. We introduce new hiring parameters that describe the dynamics of the hiring process, like the distance between the last two hirings, and the quality of the hired staff;, like the score of the best discarded candidate. While Archibald and Martínez made systematic use of analytic combinatorics techniques (Flajolet, Sedgewick, 2008) in their analysis, we use here a different approach to study the various hiring parameters related associated to the hiring process. We are able to obtain explicit formulas for the probability dis- tribution or the probability generating function of the random variables of interest in a rather direct way. The explicit nature of our results also allows a very detailed study of their asymptotic behaviour. Adding our new results to those of Archibald and Martínez leads to a very precise quantitative characterization of the hiring above the m-th best candi- date strategy. This might prove very useful in applications of the hiring process, e.g., in data stream algorithms.Postprint (published version

The hiring problem and its algorithmic applications

The hiring problem is a simple model for on-line decision-making under uncertainty, recently introduced in the literature. Despite some related work dates back to 2000, the name and the first extensive studies were written in 2007 and 2008. The problem has been introduced explicitly first by Broder et al. in 2008 as a natural extension to the well-known secretary problem. Soon afterwards, Archibald and Martínez in 2009 introduced a discrete (combinatorial) model of the hiring problem, where the candidates seen so far could be ranked from best to worst without the need to know their absolute quality scores. This thesis introduces an extensive study for the hiring problem under the formulation given by Archibald and Martínez, explores the connections with other on-line selection processes in the literature, and develops one interesting application of our results to the field of data streaming algorithms. In the hiring problem we are interested in the design and analysis of hiring strategies. We study in detail two hiring strategies, namely hiring above the median and hiring above the m-th best. Hiring above the median hires the first interviewed candidate then any coming candidate is hired if and only if his relative rank is better than the median rank of the already hired staff, and others are discarded. Hiring above the m-th best hires the first m candidates in the sequence, then any coming candidate is hired if and only if his relative rank is larger than the m-th best among all hired candidates, and others are discarded. For both strategies, we were able to obtain exact and asymptotic distributional results for various quantities of interest (which we call hiring parameters). Our fundamental parameter is the number of hired candidates, together with other parameters like waiting time, index of last hired candidate and distance between the last two hirings give us a clear picture of the hiring rate or the dynamics of the hiring process for the particular strategy under study. There is another group of parameters like score of last hired candidate, score of best discarded candidate and number of replacements that give us an indicator of the quality of the hired staff. For the strategy hiring above the median, we study more quantities like number of hired candidates conditioned on the first one and probability that the candidate with score q is getting hired. We study the selection rule 1/2-percentile rule introduced by Krieger et al., in 2007, and the seating plan (1/2,1) of the Chinese restaurant process (CRP) introduced by Pitman, which are very similar to hiring above the median. The connections between hiring above the m-th best and the notion of m-records, and also the seating plan (0,m) of the CRP are investigated here. We report preliminary results for the number of hired candidates for a generalization of hiring above the median; called hiring above the alpha-quantile (of the hired staff). The explicit results for the number of hired candidates enable us to design an estimator, called RECORDINALITY, for the number of distinct elements in a large sequence of data which may contain repetitions; this problem is known in the literature as cardinality estimation problem. We show that another hiring parameter, the score of best discarded candidate, can also be used to design a new cardinality estimator, which we call DISCARDINALITY. Most of the results presented here have been published or submitted for publication. The thesis leaves some open questions, as well as many promising ideas for future work. One interesting question is how to compare two different strategies; that requires a suitable definition of the notion of optimality, which is still missing in the context of the hiring problem. We are also interested in investigating other variants of the problem like probabilistic hiring strategies, that is when the hiring criteria is not deterministic, unlike all the studied strategies

Essays on Trade Costs, Supply Chain Uncertainty and CEO Compensation

This dissertation consists of two chapters that examine high managerial pay and supply chain uncertainty. Chapter 1 constructs a game-theoretic model in which high CEO pay emerges as the outcome of an arms race, with each firm paying its CEO highly to protect its competitive position against rivals who also pay highly. For an arms race to emerge, highly-paid CEOs must generate idiosyncratic, privately-known internal effects on profit, and CEO pay disparities must also generate asymmetric profit differences from external effects beyond the simple differences in pay. If the distribution of internal effects satisfies a key uniformity condition, an arms race emerges as the only equilibrium of the game. Chapter 2 examines the impact of supply chain uncertainty and ordering costs on trade. Importers hold safety stock to hedge against delays in delivery. An increase in supply chain uncertainty raises safety stocks, increases inventory costs, and reduces imports from locations with high delivery time uncertainty. An increase in order costs reduces a firm\u27s shipping frequency and increases average inventory holding cost for the firm\u27s base inventory stock. As a result, firms import less from locations with high ordering costs to reduce average inventory holding costs. Detailed data on actual and expected arrival times of vessels at U.S. ports serve to measure supply chain uncertainty consistent with the theory. Combined with detailed data on U.S. imports, freight charges and unit values, a 10 percent increase in supply-chain uncertainty lowers imports by as much as 3.7 percent. This is evidence that delivery uncertainty imposes a cost on imports according to the management of safety stocks. A one percent increase in ordering costs lowers imports by as much as 1.2 percent. Ordering costs impact the intensive margin of trade due to the management of base inventory stocks

The Hiring Problem: An Analytic and Experimental Study

When a small, start-up company intends to grow, it has to hire employees. Because the company requires high quality staff, the employer has to interview a lot of candidates and thus, she may take long time to collect the required staff. Of course, there is another important demand which is the time taken by the hiring process which is required to be as short as possible or the company’s rate of growth which is required to go as quickly as possible. So, the company has two main demands and needs to achieve balance between them. This is an intuitive idea about hiring from which this case study problem bears its name. The hiring problem is just an abstract model of the real hiring process. It is clear that the hiring problem will not cover every aspect of real hiring processes but it investigates some important parameters under a simplified mathematical model. On the other hand, the statement of the problem - as we will see- will give a general mathematical question with many possible applications; it is relevant in many instances where one must make decisions under uncertainty

Yearning for Lake Wobegon: The Quest for the Best Test at the Expense of the Best Education

This article first will outline the various tests or assessments. Next, recent federal and state mandates for standardized testing of elementary school children will be examined. Then, the educational literature will be reviewed to expose the dangers of testing, particularly in the early grades. Finally, I will urge that the Clinton administration, Congress, and the states step back from this manner of securing educational adequacy. These governmental policies are rooted in the stated intention of guaranteeing that all schools are doing right by all of the children. However, early testing fosters the opposite result-educational inequity through tracking, retention, and the early creation of a racial and class caste system. Furthermore, widespread testing of the type advocated by the federal government is an expensive proposition. Given the negative effects of testing, I advocate the use of these funds in other ways to address children\u27s real educational needs

Interpersonal Behavior Traits and Their Relationship to Administrator-to-Teacher Feedback: A Quantitative Study

The purpose of this study was to explore what, if any, relationship exists between the interpersonal behavior traits held by administrators and the quality of the feedback they provide to teachers. The Interpersonal Behavior Survey (IBS) was used to develop interpersonal behavior profiles for all the school-based administrators from a moderately sized school district who consented to participate in the study. Additionally, the comments submitted as feedback to teachers by the participating administrators were reviewed and scored using a rubric. Multiple regression analysis was performed to determine what, if any, relationship exists between the traits measured by the IBS and feedback quality. The IBS is divided into four scales: validity, assertiveness, aggressiveness, and relationship. These groupings were used to formulate the four research questions that guided this study: (1) what, if any, relationship exists between assertiveness traits and feedback quality, (2) what, if any, relationship exists between aggressiveness traits and feedback quality, (3) what, if any, relationship exists between relationship traits and feedback quality, and (4) what, if any, relationship exists between scores above the cut-off for reliability on any of the three validity scales and the quality of feedback given. No significant relationship was found to exist between any of the four IBS scale groupings and feedback quality; however, power analysis showed the lack of significance observed in this study could be due to the size of the population and not a true lack of significance. The study did find a significant relationship between age and years of experience in administration and feedback quality. This study is valuable in that it contributes to the conversation regarding teacher effectiveness ratings, feedback, and sheds light on the role interpersonal behavior traits held by the administrator play in the feedback giving process. This study suggests there is reason to continue exploring the important role conflict avoidance may play in teacher evaluation and teacher effectiveness ratings