Statistical Arbitrage Mining for Display Advertising

We study and formulate arbitrage in display advertising. Real-Time Bidding (RTB) mimics stock spot exchanges and utilises computers to algorithmically buy display ads per impression via a real-time auction. Despite the new automation, the ad markets are still informationally inefficient due to the heavily fragmented marketplaces. Two display impressions with similar or identical effectiveness (e.g., measured by conversion or click-through rates for a targeted audience) may sell for quite different prices at different market segments or pricing schemes. In this paper, we propose a novel data mining paradigm called Statistical Arbitrage Mining (SAM) focusing on mining and exploiting price discrepancies between two pricing schemes. In essence, our SAMer is a meta-bidder that hedges advertisers' risk between CPA (cost per action)-based campaigns and CPM (cost per mille impressions)-based ad inventories; it statistically assesses the potential profit and cost for an incoming CPM bid request against a portfolio of CPA campaigns based on the estimated conversion rate, bid landscape and other statistics learned from historical data. In SAM, (i) functional optimisation is utilised to seek for optimal bidding to maximise the expected arbitrage net profit, and (ii) a portfolio-based risk management solution is leveraged to reallocate bid volume and budget across the set of campaigns to make a risk and return trade-off. We propose to jointly optimise both components in an EM fashion with high efficiency to help the meta-bidder successfully catch the transient statistical arbitrage opportunities in RTB. Both the offline experiments on a real-world large-scale dataset and online A/B tests on a commercial platform demonstrate the effectiveness of our proposed solution in exploiting arbitrage in various model settings and market environments.Comment: In the proceedings of the 21st ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2015

Beta lives - some statistical perspectives on the capital asset pricing model

This note summarizes some technical issues relevant to the use of the idea of excess return in empirical modelling. We cover the case where the aim is to construct a measure of expected return on an asset and a model of the CAPM type is used. We review some of the problems and show examples where the basic CAPM may be used to develop other results which relate the expected returns on assets both to the expected return on the market and other factors

Multiscaled Cross-Correlation Dynamics in Financial Time-Series

The cross correlation matrix between equities comprises multiple interactions between traders with varying strategies and time horizons. In this paper, we use the Maximum Overlap Discrete Wavelet Transform to calculate correlation matrices over different timescales and then explore the eigenvalue spectrum over sliding time windows. The dynamics of the eigenvalue spectrum at different times and scales provides insight into the interactions between the numerous constituents involved. Eigenvalue dynamics are examined for both medium and high-frequency equity returns, with the associated correlation structure shown to be dependent on both time and scale. Additionally, the Epps effect is established using this multivariate method and analyzed at longer scales than previously studied. A partition of the eigenvalue time-series demonstrates, at very short scales, the emergence of negative returns when the largest eigenvalue is greatest. Finally, a portfolio optimization shows the importance of timescale information in the context of risk management

Managing Risk of Bidding in Display Advertising

In this paper, we deal with the uncertainty of bidding for display advertising. Similar to the financial market trading, real-time bidding (RTB) based display advertising employs an auction mechanism to automate the impression level media buying; and running a campaign is no different than an investment of acquiring new customers in return for obtaining additional converted sales. Thus, how to optimally bid on an ad impression to drive the profit and return-on-investment becomes essential. However, the large randomness of the user behaviors and the cost uncertainty caused by the auction competition may result in a significant risk from the campaign performance estimation. In this paper, we explicitly model the uncertainty of user click-through rate estimation and auction competition to capture the risk. We borrow an idea from finance and derive the value at risk for each ad display opportunity. Our formulation results in two risk-aware bidding strategies that penalize risky ad impressions and focus more on the ones with higher expected return and lower risk. The empirical study on real-world data demonstrates the effectiveness of our proposed risk-aware bidding strategies: yielding profit gains of 15.4% in offline experiments and up to 17.5% in an online A/B test on a commercial RTB platform over the widely applied bidding strategies

Random Matrix Theory Analysis of Cross Correlations in Financial Markets

We confirm universal behaviors such as eigenvalue distribution and spacings predicted by Random Matrix Theory (RMT) for the cross correlation matrix of the daily stock prices of Tokyo Stock Exchange from 1993 to 2001, which have been reported for New York Stock Exchange in previous studies. It is shown that the random part of the eigenvalue distribution of the cross correlation matrix is stable even when deterministic correlations are present. Some deviations in the small eigenvalue statistics outside the bounds of the universality class of RMT are not completely explained with the deterministic correlations as proposed in previous studies. We study the effect of randomness on deterministic correlations and find that randomness causes a repulsion between deterministic eigenvalues and the random eigenvalues. This is interpreted as a reminiscent of ``level repulsion'' in RMT and explains some deviations from the previous studies observed in the market data. We also study correlated groups of issues in these markets and propose a refined method to identify correlated groups based on RMT. Some characteristic differences between properties of Tokyo Stock Exchange and New York Stock Exchange are found.Comment: RevTex, 17 pages, 8 figure

Portfolio Optimization and the Random Magnet Problem

Diversification of an investment into independently fluctuating assets reduces its risk. In reality, movement of assets are are mutually correlated and therefore knowledge of cross--correlations among asset price movements are of great importance. Our results support the possibility that the problem of finding an investment in stocks which exposes invested funds to a minimum level of risk is analogous to the problem of finding the magnetization of a random magnet. The interactions for this ``random magnet problem'' are given by the cross-correlation matrix {\bf \sf C} of stock returns. We find that random matrix theory allows us to make an estimate for {\bf \sf C} which outperforms the standard estimate in terms of constructing an investment which carries a minimum level of risk.Comment: 12 pages, 4 figures, revte

Risk-Seeking versus Risk-Avoiding Investments in Noisy Periodic Environments

We study the performance of various agent strategies in an artificial investment scenario. Agents are equipped with a budget, $x(t)$, and at each time step invest a particular fraction, $q(t)$, of their budget. The return on investment (RoI), $r(t)$, is characterized by a periodic function with different types and levels of noise. Risk-avoiding agents choose their fraction $q(t)$ proportional to the expected positive RoI, while risk-seeking agents always choose a maximum value $q_{max}$ if they predict the RoI to be positive ("everything on red"). In addition to these different strategies, agents have different capabilities to predict the future $r(t)$, dependent on their internal complexity. Here, we compare 'zero-intelligent' agents using technical analysis (such as moving least squares) with agents using reinforcement learning or genetic algorithms to predict $r(t)$. The performance of agents is measured by their average budget growth after a certain number of time steps. We present results of extensive computer simulations, which show that, for our given artificial environment, (i) the risk-seeking strategy outperforms the risk-avoiding one, and (ii) the genetic algorithm was able to find this optimal strategy itself, and thus outperforms other prediction approaches considered.Comment: 27 pp. v2 with minor corrections. See http://www.sg.ethz.ch for more inf

Ground-state energies, densities and momentum distributions in closed-shell nuclei calculated within a cluster expansion approach and realistic interactions

A linked cluster expansion suitable for the treatment of ground-state properties of complex nuclei, as well as of various particle-nucleus scattering processes, has been used to calculate the ground-state energy, density and momentum distribution of 16-O and 40-Ca using realistic interactions. First of all, a benchmark calculation for the ground-state energy has been performed using the truncated V8' potential, and consisting in the comparison of our results with the ones obtained by the Fermi Hypernetted Chain approach, adopting in both cases the same mean field wave functions and the same correlation functions. The results exhibited a nice agreement between the two methods. Therefore, the approach has been applied to the calculation of the ground-state energy, density and momentum distributions of 16-O and 40-Ca using the full V8' potential, finding again a satisfactory agreement with the results based on more advanced approaches where higher order cluster contributions are taken into account. It appears therefore that the cluster expansion approach can provide accurate approximations for various diagonal and non diagonal density matrices, so that it could be used for a reliable evaluation of nuclear effects in various medium and high energy scattering processes off nuclear targets. The developed approach can be readily generalized to the treatment of Glauber type final state interaction effects in inclusive, semi-inclusive and exclusive processes off nuclei at medium and high energies.Comment: 42 pages, 18 figure

Noise Dressing of Financial Correlation Matrices

We show that results from the theory of random matrices are potentially of great interest to understand the statistical structure of the empirical correlation matrices appearing in the study of price fluctuations. The central result of the present study is the remarkable agreement between the theoretical prediction (based on the assumption that the correlation matrix is random) and empirical data concerning the density of eigenvalues associated to the time series of the different stocks of the S&P500 (or other major markets). In particular the present study raises serious doubts on the blind use of empirical correlation matrices for risk management.Comment: Latex (Revtex) 3 pp + 2 postscript figures (in-text

Data clustering and noise undressing for correlation matrices

We discuss a new approach to data clustering. We find that maximum likelihood leads naturally to an Hamiltonian of Potts variables which depends on the correlation matrix and whose low temperature behavior describes the correlation structure of the data. For random, uncorrelated data sets no correlation structure emerges. On the other hand for data sets with a built-in cluster structure, the method is able to detect and recover efficiently that structure. Finally we apply the method to financial time series, where the low temperature behavior reveals a non trivial clustering.Comment: 8 pages, 5 figures, completely rewritten and enlarged version of cond-mat/0003241. Submitted to Phys. Rev.