Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models

Is neural IR mostly hype? In a recent SIGIR Forum article, Lin expressed skepticism that neural ranking models were actually improving ad hoc retrieval effectiveness in limited data scenarios. He provided anecdotal evidence that authors of neural IR papers demonstrate "wins" by comparing against weak baselines. This paper provides a rigorous evaluation of those claims in two ways: First, we conducted a meta-analysis of papers that have reported experimental results on the TREC Robust04 test collection. We do not find evidence of an upward trend in effectiveness over time. In fact, the best reported results are from a decade ago and no recent neural approach comes close. Second, we applied five recent neural models to rerank the strong baselines that Lin used to make his arguments. A significant improvement was observed for one of the models, demonstrating additivity in gains. While there appears to be merit to neural IR approaches, at least some of the gains reported in the literature appear illusory.Comment: Published in the Proceedings of the 42nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019

Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search

Despite substantial interest in applications of neural networks to information retrieval, neural ranking models have only been applied to standard ad hoc retrieval tasks over web pages and newswire documents. This paper proposes MP-HCNN (Multi-Perspective Hierarchical Convolutional Neural Network) a novel neural ranking model specifically designed for ranking short social media posts. We identify document length, informal language, and heterogeneous relevance signals as features that distinguish documents in our domain, and present a model specifically designed with these characteristics in mind. Our model uses hierarchical convolutional layers to learn latent semantic soft-match relevance signals at the character, word, and phrase levels. A pooling-based similarity measurement layer integrates evidence from multiple types of matches between the query, the social media post, as well as URLs contained in the post. Extensive experiments using Twitter data from the TREC Microblog Tracks 2011--2014 show that our model significantly outperforms prior feature-based as well and existing neural ranking models. To our best knowledge, this paper presents the first substantial work tackling search over social media posts using neural ranking models.Comment: AAAI 2019, 10 page

Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching

Leveraging on the recent developments in convolutional neural networks (CNNs), matching dense correspondence from a stereo pair has been cast as a learning problem, with performance exceeding traditional approaches. However, it remains challenging to generate high-quality disparities for the inherently ill-posed regions. To tackle this problem, we propose a novel cascade CNN architecture composing of two stages. The first stage advances the recently proposed DispNet by equipping it with extra up-convolution modules, leading to disparity images with more details. The second stage explicitly rectifies the disparity initialized by the first stage; it couples with the first-stage and generates residual signals across multiple scales. The summation of the outputs from the two stages gives the final disparity. As opposed to directly learning the disparity at the second stage, we show that residual learning provides more effective refinement. Moreover, it also benefits the training of the overall cascade network. Experimentation shows that our cascade residual learning scheme provides state-of-the-art performance for matching stereo correspondence. By the time of the submission of this paper, our method ranks first in the KITTI 2015 stereo benchmark, surpassing the prior works by a noteworthy margin.Comment: Accepted at ICCVW 2017. The first two authors contributed equally to this pape

Protected Methionine and Heat-Treated Soybean Meal for High-Producing Dairy Cows

The effectiveness of a rumen-protected methionine preparation was studied as an amino acid source for high-producing dairy cows during wk 4 through 16 postpartum. Twenty-four Holstein cows (14 primiparous and 10 multiparous) were fed concentrate containing heat-treated· soybean meal without or with 50 g/cow/day of added ruminally protected methionine product which provided 15 g of added DL-methionine daily. Cows were fed 16% crude protein mixed diets containing 30% of dry matter as corn silage, 15% as alfalfa hay, and 55% as concentrate. Milk production and composition were adjusted for pretreatment values (3 wk postpartum) by analysis of covariance. Adjusted milk yields (34.6 and 33.1 kg/day) were higher for cows fed heated soybean meal, but this difference was accounted for by higher pretreatment production (32.6 and 36.9 kg/day) of multiparous cows fed supplemental methionine. Production of 4% fat-corrected milk (28.5 and 27.6 kg/day) and solids-corrected milk (29.0 and 28.5 kg/day) was similar for cows fed both diets. Percentages of fat (2.81 and 2.92) and protein (2.88 and 2.92) were similar, while total solids (11.49 and 12.69) and solids-not-fat (8.68 and 8.77) were higher when cows were fed supplemental methionine. Milk protein percent (2.89 and 2.99) and milk protein production (.97 and 1.00 kg/day) were increased for primiparous cows fed supplemental methionine. Fatty acid composition in milk was similar. Dry matter intakes (20.2 and 21.0 kg/day) were higher especially in multiparous cows (21.5 and 23.8 kg/day) when fed supplemental methionine. Body weights (602 and 598 kg) and body weight changes were similar for the two treatments. Ruminal pH, volatile fatty acids, and ammonia, as well as blood serum urea and glucose were generally unaffected by methionine supplementation. Concentrations of methionine in arterial and venous plasma were elevated slightly when fed additional methionine, but the first in arterial and venous plasma were elevated fed additional methionine, but the first limiting amino acid for milk production, as calculated by several methods, was not changed by feeding supplemental ruminally protected methionine

Judea Captured

This coin is a part of the “Judea Captured” collection minted in Caesarea. Through thorough imaging and the decoding of Greek inscriptions we believe that we can prove the significance of the coin in correlation to the First Jewish War in battle against the Romans. We believe that it celebrates the victory of the Romans during the First Jewish War (66 – 69 AD) , under the Emperor Vespasian (68 – 79 AD). But the question does arise about the significance of certain symbols; especially the palm tree and crown made of palm leaves and how they relate to the War as well

Repeatability Corner Cases in Document Ranking: The Impact of Score Ties

Document ranking experiments should be repeatable. However, the interaction between multi-threaded indexing and score ties during retrieval may yield non-deterministic rankings, making repeatability not as trivial as one might imagine. In the context of the open-source Lucene search engine, score ties are broken by internal document ids, which are assigned at index time. Due to multi-threaded indexing, which makes experimentation with large modern document collections practical, internal document ids are not assigned consistently between different index instances of the same collection, and thus score ties are broken unpredictably. This short paper examines the effectiveness impact of such score ties, quantifying the variability that can be attributed to this phenomenon. The obvious solution to this non-determinism and to ensure repeatable document ranking is to break score ties using external collection document ids. This approach, however, comes with measurable efficiency costs due to the necessity of consulting external identifiers during query evaluation.Comment: Published in the Proceedings of the 42nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019