Learning to Diversify Web Search Results with a Document Repulsion Model

Search diversification (also called diversity search), is an important approach to tackling the query ambiguity problem in information retrieval. It aims to diversify the search results that are originally ranked according to their probabilities of relevance to a given query, by re-ranking them to cover as many as possible different aspects (or subtopics) of the query. Most existing diversity search models heuristically balance the relevance ranking and the diversity ranking, yet lacking an efficient learning mechanism to reach an optimized parameter setting. To address this problem, we propose a learning-to-diversify approach which can directly optimize the search diversification performance (in term of any effectiveness metric). We first extend the ranking function of a widely used learning-to-rank framework, i.e., LambdaMART, so that the extended ranking function can correlate relevance and diversity indicators. Furthermore, we develop an effective learning algorithm, namely Document Repulsion Model (DRM), to train the ranking function based on a Document Repulsion Theory (DRT). DRT assumes that two result documents covering similar query aspects (i.e., subtopics) should be mutually repulsive, for the purpose of search diversification. Accordingly, the proposed DRM exerts a repulsion force between each pair of similar documents in the learning process, and includes the diversity effectiveness metric to be optimized as part of the loss function. Although there have been existing learning based diversity search methods, they often involve an iterative sequential selection process in the ranking process, which is computationally complex and time consuming for training, while our proposed learning strategy can largely reduce the time cost. Extensive experiments are conducted on the TREC diversity track data (2009, 2010 and 2011). The results demonstrate that our model significantly outperforms a number of baselines in terms of effectiveness and robustness. Further, an efficiency analysis shows that the proposed DRM has a lower computational complexity than the state of the art learning-to-diversify methods

Biogeography of Endemic Dragonflies of the Ozark-Ouachita Interior Highlands

A common pattern across many taxonomic groups is that relatively few species are widespread while the majority are restricted in their geographic ranges. Such species distributions are used to inform conservation status, which poses unique challenges for rare or cryptic species. Further, priority status is often designated within geopolitical boundaries, which may include only a portion of a species range. This, coupled with lack of distributional data, has resulted in species being designated as apparently rare throughout some portions of their range, which may not accurately reflect their overall conservation need. The Interior Highlands region of the central United States harbors a rich diversity of flora and fauna, many of which are regional endemics. Among these are four dragonfly species considered Species of Greatest Conservation Need: Ouachita spiketail (Cordulegaster talaria), Ozark Emerald (Somatochlora ozarkensis), Westfall’s snaketail (Ophiogomphus westfalli), and Ozark clubtail (Gomphurus ozarkensis). I combined species distribution modeling with field surveys to better understand the current biogeography for the two species with ample presence data (S. ozarkensis and G. ozarkensis). Additionally, models were used to project species’ distributions under two climate change scenarios of differing severity. To assess reliability of model predictions, I used two machine learning algorithms commonly used with limited, presence-only data. Current areas of suitability predicted by both algorithms largely overlapped for each species. An analysis of variable contribution showed congruence in important environmental predictors between models. Field validation of these models resulted in new detections for both species showing their utility in guiding future surveys. Future projections across two climate change scenarios showed the importance of maintaining current suitable areas as these will continue to be strongholds for these species under climate change

How Many Topics? Stability Analysis for Topic Models

Topic modeling refers to the task of discovering the underlying thematic structure in a text corpus, where the output is commonly presented as a report of the top terms appearing in each topic. Despite the diversity of topic modeling algorithms that have been proposed, a common challenge in successfully applying these techniques is the selection of an appropriate number of topics for a given corpus. Choosing too few topics will produce results that are overly broad, while choosing too many will result in the "over-clustering" of a corpus into many small, highly-similar topics. In this paper, we propose a term-centric stability analysis strategy to address this issue, the idea being that a model with an appropriate number of topics will be more robust to perturbations in the data. Using a topic modeling approach based on matrix factorization, evaluations performed on a range of corpora show that this strategy can successfully guide the model selection process.Comment: Improve readability of plots. Add minor clarification

Predicting global habitat suitability for stony corals on seamounts

Aim Globally, species distribution patterns in the deep sea are poorly resolved, with spatial coverage being sparse for most taxa and true absence data missing. Increasing human impacts on deep-sea ecosystems mean that reaching a better understanding of such patterns is becoming more urgent. Cold-water stony corals (Order Scleractinia) form structurally complex habitats (dense thickets or reefs) that can support a diversity of other associated fauna. Despite their widely accepted ecological importance, records of scleractinian corals on seamounts are patchy and simply not available for most of the global ocean. The objective of this paper is to model the global distribution of suitable habitat for stony corals on seamounts. Location Seamounts worldwide. Methods We compiled a database containing all accessible records of scleractinian corals on seamounts. Two modelling approaches developed for presence-only data were used to predict global habitat suitability for seamount scleractinians: maximum entropy modelling (Maxent) and environmental niche factor analysis (ENFA). We generated habitat-suitability maps and used a cross-validation process with a threshold-independent metric to evaluate the performance of the models. Results Both models performed well in cross-validation, although the Maxent method consistently outperformed ENFA. Highly suitable habitat for seamount stony corals was predicted to occur at most modelled depths in the North Atlantic, and in a circumglobal strip in the Southern Hemisphere between 20° and 50° S and shallower than around 1500 m. Seamount summits in most other regions appeared much less likely to provide suitable habitat, except for small near-surface patches. The patterns of habitat suitability largely reflect current biogeographical knowledge. Environmental variables positively associated with high predicted habitat suitability included the aragonite saturation state, and oxygen saturation and concentration. By contrast, low levels of dissolved inorganic carbon, nitrate, phosphate and silicate were associated with high predicted suitability. High correlation among variables made assessing individual drivers difficult. Main conclusions Our models predict environmental conditions likely to play a role in determining large-scale scleractinian coral distributions on seamounts, and provide a baseline scenario on a global scale. These results present a first-order hypothesis that can be tested by further sampling. Given the high vulnerability of cold-water corals to human impacts, such predictions are crucial tools in developing worldwide conservation and management strategies for seamount ecosystems. © 2009 Blackwell Publishing Ltd

Predicting expected TCP throughput using genetic algorithm

Predicting the expected throughput of TCP is important for several aspects such as e.g. determining handover criteria for future multihomed mobile nodes or determining the expected throughput of a given MPTCP subflow for load-balancing reasons. However, this is challenging due to time varying behavior of the underlying network characteristics. In this paper, we present a genetic-algorithm-based prediction model for estimating TCP throughput values. Our approach tries to find the best matching combination of mathematical functions that approximate a given time series that accounts for the TCP throughput samples using genetic algorithm. Based on collected historical datapoints about measured TCP throughput samples, our algorithm estimates expected throughput over time. We evaluate the quality of the prediction using different selection and diversity strategies for creating new chromosomes. Also, we explore the use of different fitness functions in order to evaluate the goodness of a chromosome. The goal is to show how different tuning on the genetic algorithm may have an impact on the prediction. Using extensive simulations over several TCP throughput traces, we find that the genetic algorithm successfully finds reasonable matching mathematical functions that allow to describe the TCP sampled throughput values with good fidelity. We also explore the effectiveness of predicting time series throughput samples for a given prediction horizon and estimate the prediction error and confidence.Peer ReviewedPostprint (author's final draft

Variation in Spatial Predictions Among Species Distribution Modeling Methods

<p>Prediction maps produced by species distribution models (SDMs) influence decision-making in resource management or designation of land in conservation planning. Many studies have compared the prediction accuracy of different SDM modeling methods, but few have quantified the similarity among prediction maps. There has also been little systematic exploration of how the relative importance of different predictor variables varies among model types. Our objective was to expand the evaluation of SDM performance for 45 plant species in southern California to better understand how map predictions vary among model types, and to explain what factors may affect spatial correspondence, including the selection and relative importance of different environmental variables. Four types of models were tested. Correlation among maps was highest between generalized linear models (GLMs) and generalized additive models (GAMs) and lowest between classification trees and GAMs or GLMs. Correlation between Random Forests (RFs) and GAMs was the same as between RFs and classification trees. Spatial correspondence among maps was influenced the most by model prediction accuracy (AUC) and species prevalence; map correspondence was highest when accuracy was high and prevalence was intermediate. Species functional type and the selection of climate variables also influenced map correspondence. For most (but not all) species, climate variables were more important than terrain or soil in predicting their distributions. Environmental variable selection varied according to modeling method, but the largest differences were between RFs and GLMs or GAMs. Although prediction accuracy was equal for GLMs, GAMs, and RFs, the differences in spatial predictions suggest that it may be important to evaluate the results of more than one model to estimate a range of spatial uncertainty before making planning decisions based on map outputs. This may be particularly important if models have low accuracy or if species prevalence is not intermediate.</p>