Balancing Speed and Quality in Online Learning to Rank for Information Retrieval

In Online Learning to Rank (OLTR) the aim is to find an optimal ranking model by interacting with users. When learning from user behavior, systems must interact with users while simultaneously learning from those interactions. Unlike other Learning to Rank (LTR) settings, existing research in this field has been limited to linear models. This is due to the speed-quality tradeoff that arises when selecting models: complex models are more expressive and can find the best rankings but need more user interactions to do so, a requirement that risks frustrating users during training. Conversely, simpler models can be optimized on fewer interactions and thus provide a better user experience, but they will converge towards suboptimal rankings. This tradeoff creates a deadlock, since novel models will not be able to improve either the user experience or the final convergence point, without sacrificing the other. Our contribution is twofold. First, we introduce a fast OLTR model called Sim-MGD that addresses the speed aspect of the speed-quality tradeoff. Sim-MGD ranks documents based on similarities with reference documents. It converges rapidly and, hence, gives a better user experience but it does not converge towards the optimal rankings. Second, we contribute Cascading Multileave Gradient Descent (C-MGD) for OLTR that directly addresses the speed-quality tradeoff by using a cascade that enables combinations of the best of two worlds: fast learning and high quality final convergence. C-MGD can provide the better user experience of Sim-MGD while maintaining the same convergence as the state-of-the-art MGD model. This opens the door for future work to design new models for OLTR without having to deal with the speed-quality tradeoff.Comment: CIKM 2017, Proceedings of the 2017 ACM on Conference on Information and Knowledge Managemen

Easy Innovation and the Iron Cage: Best Practice, Benchmarking, Ranking, and the Management of Organizational Creativity

The use of what came to be known as best practices, benchmarking, and ranking, which took corporate America by storm in the 1980s as a method for managing innovation, has seeped into government and nonprofit organizations in the intervening years. In fact, as H. George Frederickson demonstrates in this Kettering Foundation occasional paper, these practices have proven to be counterproductive both in the business and the public sector. Frederickson suggests, instead, a more flexible, less directive, model he calls "sustained innovation." He offers abundant evidence that this model is more effective in producing organizational effectiveness

Venture capital and risk in high-technology enterprises

We find UK investors and entrepreneurs are significantly concordant in rankings of investments and key factors for risk but significantly discordant on risk classes. Investors emphasise agency risk (e.g., motivation, empowerment, alignment), and entrepreneurs emphasise business risk (e.g., market opportunities)

An Analysis of the Selection of Arbitrators

This paper analyses data on union and employer rankings of different panels of arbitrators in an actual arbitration system. A random utility model of bargainer preferences is developed and estimated. The estimates indicate that unions and employers have similar preferences, in favor of lawyers, more experienced arbitrators, and arbitrators who seem to have previously favored their side. Alternative rankings models, which are estimated to test whether bargainers rank arbitrators strategically, reveal no evidence of strategic behavior.

Robust Tests in Genome-Wide Scans under Incomplete Linkage Disequilibrium

Under complete linkage disequilibrium (LD), robust tests often have greater power than Pearson's chi-square test and trend tests for the analysis of case-control genetic association studies. Robust statistics have been used in candidate-gene and genome-wide association studies (GWAS) when the genetic model is unknown. We consider here a more general incomplete LD model, and examine the impact of penetrances at the marker locus when the genetic models are defined at the disease locus. Robust statistics are then reviewed and their efficiency and robustness are compared through simulations in GWAS of 300,000 markers under the incomplete LD model. Applications of several robust tests to the Wellcome Trust Case-Control Consortium [Nature 447 (2007) 661--678] are presented.Comment: Published in at http://dx.doi.org/10.1214/09-STS314 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org