The Role and Relevance of Rankings in Higher Education Policymaking

Explores the rise of college rankings, similarities and differences from postsecondary assessment efforts, and factors behind their limited relevance to policy such as their effect on institutional behaviors. Recommends ways to enhance policy relevance

College and University Ranking Systems: Global Perspectives and American Challenges

Examines how higher education ranking systems function, how other countries use ranking systems, and the impact of college rankings in the United States on student access, choice, and opportunity

Fidelity-Weighted Learning

Training deep neural networks requires many training samples, but in practice training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account when learning the data representation, we could get the best of both worlds. To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network (trained on the task we care about) on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher (who has access to the high-quality labels). Both student and teacher are learned from the data. We evaluate FWL on two tasks in information retrieval and natural language processing where we outperform state-of-the-art alternative semi-supervised methods, indicating that our approach makes better use of strong and weak labels, and leads to better task-dependent data representations.Comment: Published as a conference paper at ICLR 201

A new perspective on the competitiveness of nations

The capability of firms to survive and to have a competitive advantage in global markets depends on, amongst other things, the efficiency of public institutions, the excellence of educational, health and communications infrastructures, as well as on the political and economic stability of their home country. The measurement of competitiveness and strategy development is thus an important issue for policy-makers. Despite many attempts to provide objectivity in the development of measures of national competitiveness, there are inherently subjective judgments that involve, for example, how data sets are aggregated and importance weights are applied. Generally, either equal weighting is assumed in calculating a final index, or subjective weights are specified. The same problem also occurs in the subjective assignment of countries to different clusters. Developed as such, the value of these type indices may be questioned by users. The aim of this paper is to explore methodological transparency as a viable solution to problems created by existing aggregated indices. For this purpose, a methodology composed of three steps is proposed. To start, a hierarchical clustering analysis is used to assign countries to appropriate clusters. In current methods, country clustering is generally based on GDP. However, we suggest that GDP alone is insufficient for purposes of country clustering. In the proposed methodology, 178 criteria are used for this purpose. Next, relationships between the criteria and classification of the countries are determined using artificial neural networks (ANNs). ANN provides an objective method for determining the attribute/criteria weights, which are, for the most part, subjectively specified in existing methods. Finally, in our third step, the countries of interest are ranked based on weights generated in the previous step. Beyond the ranking of countries, the proposed methodology can also be used to identify those attributes that a given country should focus on in order to improve its position relative to other countries, i.e., to transition from its current cluster to the next higher one

A Novel Approach for Learning How to Automatically Match Job Offers and Candidate Profiles

Automatic matching of job offers and job candidates is a major problem for a number of organizations and job applicants that if it were successfully addressed could have a positive impact in many countries around the world. In this context, it is widely accepted that semi-automatic matching algorithms between job and candidate profiles would provide a vital technology for making the recruitment processes faster, more accurate and transparent. In this work, we present our research towards achieving a realistic matching approach for satisfactorily addressing this challenge. This novel approach relies on a matching learning solution aiming to learn from past solved cases in order to accurately predict the results in new situations. An empirical study shows us that our approach is able to beat solutions with no learning capabilities by a wide margin.Comment: 15 pages, 6 figure

Counting What Is Measured or Measuring What Counts? League Tables and Their Impact On Higher Education Institutions in England

This report investigates league tables and their impact on higher education institutions (HEIs) in England. It presents findings from two strands of research: – an analysis of five league tables selected for the study, their methodologies and the underlying data employed, and – an investigation of how higher education institutions respond to league tables generally and the extent to which they influence institutional decision-making and actions. The purpose of the research is to stimulate informed debate about the approaches and limitations of the various league tables, and greater understanding among the users and stakeholders of the implications of making decisions based on these sources of information

Combining Terrier with Apache Spark to Create Agile Experimental Information Retrieval Pipelines

Experimentation using IR systems has traditionally been a procedural and laborious process. Queries must be run on an index, with any parameters of the retrieval models suitably tuned. With the advent of learning-to-rank, such experimental processes (including the appropriate folding of queries to achieve cross-fold validation) have resulted in complicated experimental designs and hence scripting. At the same time, machine learning platforms such as Scikit Learn and Apache Spark have pioneered the notion of an experimental pipeline , which naturally allows a supervised classification experiment to be expressed a series of stages, which can be learned or transformed. In this demonstration, we detail Terrier-Spark, a recent adaptation to the Terrier Information Retrieval platform which permits it to be used within the experimental pipelines of Spark. We argue that this (1) provides an agile experimental platform for information retrieval, comparable to that enjoyed by other branches of data science; (2) aids research reproducibility in information retrieval by facilitating easily-distributable notebooks containing conducted experiments; and (3) facilitates the teaching of information retrieval experiments in educational environments