Practical Batch Bayesian Sampling Algorithms for Online Adaptive Traffic Experimentation

Online controlled experiments have emerged as industry gold standard for assessing new web features. As new web algorithms proliferate, experimentation platform faces an increasing demand on the velocity of online experiments, which encourages adaptive traffic testing methods to speed up identifying best variant by efficiently allocating traffic. This paper proposed four Bayesian batch bandit algorithms (NB-TS, WB-TS, NB-TTTS, WB-TTTS) for eBay's experimentation platform, using summary batch statistics of a goal metric without incurring new engineering technical debts. The novel WB-TTTS, in particular, demonstrates as an efficient, trustworthy and robust alternative to fixed horizon A/B testing. Another novel contribution is to bring trustworthiness of best arm identification algorithms into evaluation criterion and highlight the existence of severe false positive inflation with equivalent best arms. To gain the trust of experimenters, experimentation platform must consider both efficiency and trustworthiness; However, to the best of authors' knowledge, trustworthiness as an important topic is rarely discussed. This paper shows that Bayesian bandits without neutral posterior reshaping, particularly naive Thompson sampling (NB-TS), are untrustworthy because they can always identify an arm as the best from equivalent best arms. To restore trustworthiness, a novel finding uncovers connections between convergence distribution of posterior optimal probabilities of equivalent best arms and neutral posterior reshaping, which controls false positives. Lastly, this paper presents lessons learned from eBay's experience, as well as thorough evaluations. We hope this work is useful to other industrial practitioners and inspires academic researchers interested in the trustworthiness of adaptive traffic experimentation

Moving Metric Detection and Alerting System at eBay

At eBay, there are thousands of product health metrics for different domain teams to monitor. We built a two-phase alerting system to notify users with actionable alerts based on anomaly detection and alert retrieval. In the first phase, we developed an efficient anomaly detection algorithm, called Moving Metric Detector (MMD), to identify potential alerts among metrics with distribution agnostic criteria. In the second alert retrieval phase, we built additional logic with feedbacks to select valid actionable alerts with point-wise ranking model and business rules. Compared with other trend and seasonality decomposition methods, our decomposer is faster and better to detect anomalies in unsupervised cases. Our two-phase approach dramatically improves alert precision and avoids alert spamming in eBay production.Comment: The work is oral presented on the AAAI-20 Workshop on Cloud Intelligence, 202

Artificial Error Generation with Machine Translation and Syntactic Patterns.

Shortage of available training data is holding back progress in the area of automated error detection. This paper investigates two alternative methods for artificially generating writing errors, in order to create additional resources. We propose treating error generation as a machine translation task, where grammatically correct text is translated to contain errors. In addition, we explore a system for extracting textual patterns from an annotated corpus, which can then be used to insert errors into grammatically correct sentences. Our experiments show that the inclusion of artificially generated errors significantly improves error detection accuracy on both FCE and CoNLL 2014 datasets.Comment: The 12th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2017

(Teff,log g,[Fe/H]) Classification of Low-Resolution Stellar Spectra using Artificial Neural Networks

New generation large-aperture telescopes, multi-object spectrographs, and large format detectors are making it possible to acquire very large samples of stellar spectra rapidly. In this context, traditional star-by-star spectroscopic analysis are no longer practical. New tools are required that are capable of extracting quickly and with reasonable accuracy important basic stellar parameters coded in the spectra. Recent analyses of Artificial Neural Networks (ANNs) applied to the classification of astronomical spectra have demonstrated the ability of this concept to derive estimates of temperature and luminosity. We have adapted the back-propagation ANN technique developed by von Hippel et al. (1994) to predict effective temperatures, gravities and overall metallicities from spectra with resolving power ~ 2000 and low signal-to-noise ratio. We show that ANN techniques are very effective in executing a three-parameter (Teff,log g,[Fe/H]) stellar classification. The preliminary results show that the technique is even capable of identifying outliers from the training sample.Comment: 6 pages, 3 figures (5 files); to appear in the proceedings of the 11th Cambridge Workshop on Cool Stars, Stellar Systems and the Sun, held on Tenerife (Spain), October 1999; also available at http://hebe.as.utexas.ed

Grammatical Error Correction: A Survey of the State of the Art

Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments

Density Functional Theory Study of Pt_3M Alloy Surface Segregation with Adsorbed O/OH and Pt_3Os as Catalysts for Oxygen Reduction Reaction

Using quantum mechanics calculations, we have studied the segregation energy with adsorbed O and OH for 28 Pt_3M alloys, where M is a transition metal. The calculations found surface segregation to become energetically unfavorable for Pt_3Co and Pt_3Ni, as well as for the most other Pt binary alloys, in the presence of adsorbed O and OH. However, Pt_3Os and Pt_3Ir remain surface segregated and show the best energy preference among the alloys studied for both adsorbed species on the surface. Binding energies of various oxygen reduction reaction (ORR) intermediates on the Pt(111) and Pt_3Os(111) surfaces were calculated and analyzed. Energy barriers for different ORR steps were computed for Pt and Pt_3Os catalysts, and the rate-determining steps (RDS) were identified. It turns out that the RDS barrier for the Pt_3Os alloy catalyst is lower than the corresponding barrier for pure Pt. This result allows us to predict a better ORR performance of Pt_3Os compared to that of pure Pt

Overview of Mollisols in the world: Distribution, land use and management

Mollisols a.k.a., Black Soils or Prairie Soils make up about 916 million ha, which is 7% of the world’s ice-free land surface. Their distribution strongly correlates with native prairie ecosystems, but is not limited to them. They are most prevalent in the mid-latitudes of North America, Eurasia, and South America. In North America, they cover 200 million ha of the United States, more than 40 million ha of Canada and 50 million ha of Mexico. Across Eurasia they cover around 450 million ha, extending from the western 148 million ha in southern Russia and 34 million ha in Ukraine to the eastern 35 million ha in northeast China. They are common to South America’s Argentina and Uruguay, covering about 89 million and 13 million ha, respectively. Mollisols are often recognized as inherently productive and fertile soils. They are extensively and intensively farmed, and increasingly dedicated to cereals production, which needs significant inputs of fertilizers and tillage. Mollisols are also important soils in pasture, range and forage systems. Thus, it is not surprising that these soils are prone to soil erosion, dehumification (loss of stable aggregates and organic matter) and are suffering from anthropogenic soil acidity. Therefore, soil scientists from all of the world’s Mollisols regions are concerned about the sustainability of some of current trends in land use and agricultural practices. These same scientists recommend increasing the acreage under minimum or restricted tillage, returning plant residues and adding organic amendments such as animal manure to maintain or increase soil organic matter content, and more systematic use of chemical amendments such as agricultural limestone to replenish soil calcium reserves