Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval

Neural networks with deep architectures have demonstrated significant performance improvements in computer vision, speech recognition, and natural language processing. The challenges in information retrieval (IR), however, are different from these other application areas. A common form of IR involves ranking of documents--or short passages--in response to keyword-based queries. Effective IR systems must deal with query-document vocabulary mismatch problem, by modeling relationships between different query and document terms and how they indicate relevance. Models should also consider lexical matches when the query contains rare terms--such as a person's name or a product model number--not seen during training, and to avoid retrieving semantically related but irrelevant results. In many real-life IR tasks, the retrieval involves extremely large collections--such as the document index of a commercial Web search engine--containing billions of documents. Efficient IR methods should take advantage of specialized IR data structures, such as inverted index, to efficiently retrieve from large collections. Given an information need, the IR system also mediates how much exposure an information artifact receives by deciding whether it should be displayed, and where it should be positioned, among other results. Exposure-aware IR systems may optimize for additional objectives, besides relevance, such as parity of exposure for retrieved items and content publishers. In this thesis, we present novel neural architectures and methods motivated by the specific needs and challenges of IR tasks.Comment: PhD thesis, Univ College London (2020

What Do International Tests Really Show About U.S. Student Performance?

Evidence-based policy has been a goal of American education policymakers for at least two decades. School reformers seek data about student knowledge and skills, hoping to use this information to improve schools. One category of such evidence, international test results, has seemingly permitted comparisons of student performance in the United States with that in other countries. Such comparisons have frequently been interpreted to show that American students perform poorly when compared to students internationally. From this, reformers conclude that U.S. public education is failing and that its failure imperils America's ability to compete with other nations economically.This report, however, shows that such inferences are too glib. Comparative student performance on international tests should be interpreted with much greater care than policymakers typically give it

What are we learning from business training and entrepreneurship evaluations around the developing world?

Business training programs are a popular policy option to try to improve the performance of enterprises around the world. The last few years have seen rapid growth in the number of evaluations of these programs in developing countries. We undertake a critical review of these studies with the goal of synthesizing the emerging lessons and understanding the limitations of the existing research and the areas in which more work is needed. We find that there is substantial heterogeneity in the length, content, and types of firms participating in the training programs evaluated. Many evaluations suffer from low statistical power, measure impacts only within a year of training, and experience problems with survey attrition and measurement of firm profits and revenues. Over these short time horizons, there are relatively modest impacts of training on survivorship of existing firms, but stronger evidence that training programs help prospective owners launch new businesses more quickly. Most studies find that existing firm owners implement some of the practices taught in training, but the magnitudes of these improvements in practices are often relatively modest. Few studies find significant impacts on profits or sales, although a couple of the studies with more statistical power have done so. Some studies have also found benefits to microfinance organizations of offering training. To date there is little evidence to help guide policymakers as to whether any impacts found come from trained firms competing away sales from other businesses versus through productivity improvements, and little evidence to guide the development of the provision of training at market prices. We conclude by summarizing some directions and key questions for future studies