AUC Optimisation and Collaborative Filtering

In recommendation systems, one is interested in the ranking of the predicted items as opposed to other losses such as the mean squared error. Although a variety of ways to evaluate rankings exist in the literature, here we focus on the Area Under the ROC Curve (AUC) as it widely used and has a strong theoretical underpinning. In practical recommendation, only items at the top of the ranked list are presented to the users. With this in mind, we propose a class of objective functions over matrix factorisations which primarily represent a smooth surrogate for the real AUC, and in a special case we show how to prioritise the top of the list. The objectives are differentiable and optimised through a carefully designed stochastic gradient-descent-based algorithm which scales linearly with the size of the data. In the special case of square loss we show how to improve computational complexity by leveraging previously computed measures. To understand theoretically the underlying matrix factorisation approaches we study both the consistency of the loss functions with respect to AUC, and generalisation using Rademacher theory. The resulting generalisation analysis gives strong motivation for the optimisation under study. Finally, we provide computation results as to the efficacy of the proposed method using synthetic and real data

Trends in Mathematical Imaging and Surface Processing

Motivated both by industrial applications and the challenge of new problems, one observes an increasing interest in the field of image and surface processing over the last years. It has become clear that even though the applications areas differ significantly the methodological overlap is enormous. Even if contributions to the field come from almost any discipline in mathematics, a major role is played by partial differential equations and in particular by geometric and variational modeling and by their numerical counterparts. The aim of the workshop was to gather a group of leading experts coming from mathematics, engineering and computer graphics to cover the main developments

Multi-Step Forecast of the Implied Volatility Surface Using Deep Learning

Implied volatility is an essential input to price an option. Machine learning architectures have shown strengths in learning option pricing formulas and estimating implied volatility cross-sectionally. However, implied volatility time series forecasting is typically done using the univariate time series and often for short intervals. When a univariate implied volatility series is forecasted, important implied volatility properties such as volatility skew and the term structure are lost. More importantly, short term forecasts can’t take advantage of the long term persistence in the volatility series. The thesis attempts to bridge the gap between machine learning-based implied volatility modeling and multivariate multi-step implied volatility forecasting. The thesis contributes to the literature by modeling the entire implied volatility surface (IVS) using recurrent neural network architectures. I implement Convolutional Long Short Term Memory Neural Network (ConvLSTM) to produce multivariate and multi-step forecasts of the S&P 500 implied volatility surface. The ConvLSTM model is capable of understanding the spatiotemporal relationships between strikes and maturities (term structure), and of modeling volatility surface dynamics non-parametrically. I benchmark the ConvLSTM model against traditional multivariate time series Vector autoregression (VAR), Vector Error Correction (VEC) model, and deep learning-based Long-Short-Term Memory (LSTM) neural network. I find that the ConvLSTM significantly outperforms traditional time series models, as well as the benchmark Long Short Term Memory(LSTM) model in predicting the implied volatility surface for a 1-day, 30-day, and 90-day horizon, for out-of-the-money and at-the-money calls and puts

Topology Optimization Using Load Path and Homogenization

In this work, the connection between topology optimization and load transfer has been established. New methods for determining load paths in two dimensional structures, plates and shells are introduced. In the two-dimensional space, there are two load paths with their total derivative equal to the transferred load, their partial derivatives related to stress tensor, and satisfying equilibrium. In the presence of a body load the stress tensor can be decomposed into solenoidal and irrotational fields using Gurtin or Helmholtz decomposition. The load path is calculated using the solenoidal field. A novel method for topology optimization using load paths and total variation of different objective functions is formulated and implemented. This approach uses the total variation to minimize different objective functions, such as compliance and norm of stress subjected to equilibrium. Since the problems are convex, the optimized solution is a global optimum which is found by solving the Euler-Lagrange optimality criteria. The optimal density of a structure is derived using optimality criteria and optimized load paths. To attain the topology of the microstructure, the principal load paths that follow the optimal principal stress directions are calculated. Since the principal stress vector field is not curl free, a dilation field is multiplied to extract the curl free component of principal stress vectors. The principal vector field has singularities which are removed by an interpolation scheme that rotates the vectors by n to construct a coherent vector field. The optimal periodic rectangular microstructure is constructed using the load functions and microstructure dimensions. The advantage of this scheme is that using the load path reduces the equilibrium constraints from two to one, and the variables are reduced from three stresses to two load functions. The non-linear elliptic partial differential equations which are derived from the total variation equations (Euler-Lagrange) are solved using the Gauss- Newton method which has a quadratic convergence, speeding up the convergence towards the optimal structure

Video enhancement : content classification and model selection

The purpose of video enhancement is to improve the subjective picture quality. The field of video enhancement includes a broad category of research topics, such as removing noise in the video, highlighting some specified features and improving the appearance or visibility of the video content. The common difficulty in this field is how to make images or videos more beautiful, or subjectively better. Traditional approaches involve lots of iterations between subjective assessment experiments and redesigns of algorithm improvements, which are very time consuming. Researchers have attempted to design a video quality metric to replace the subjective assessment, but so far it is not successful. As a way to avoid heuristics in the enhancement algorithm design, least mean square methods have received considerable attention. They can optimize filter coefficients automatically by minimizing the difference between processed videos and desired versions through a training. However, these methods are only optimal on average but not locally. To solve the problem, one can apply the least mean square optimization for individual categories that are classified by local image content. The most interesting example is Kondo’s concept of local content adaptivity for image interpolation, which we found could be generalized into an ideal framework for content adaptive video processing. We identify two parts in the concept, content classification and adaptive processing. By exploring new classifiers for the content classification and new models for the adaptive processing, we have generalized a framework for more enhancement applications. For the part of content classification, new classifiers have been proposed to classify different image degradations such as coding artifacts and focal blur. For the coding artifact, a novel classifier has been proposed based on the combination of local structure and contrast, which does not require coding block grid detection. For the focal blur, we have proposed a novel local blur estimation method based on edges, which does not require edge orientation detection and shows more robust blur estimation. With these classifiers, the proposed framework has been extended to coding artifact robust enhancement and blur dependant enhancement. With the content adaptivity to more image features, the number of content classes can increase significantly. We show that it is possible to reduce the number of classes without sacrificing much performance. For the part of model selection, we have introduced several nonlinear filters to the proposed framework. We have also proposed a new type of nonlinear filter, trained bilateral filter, which combines both advantages of the original bilateral filter and the least mean square optimization. With these nonlinear filters, the proposed framework show better performance than with linear filters. Furthermore, we have shown a proof-of-concept for a trained approach to obtain contrast enhancement by a supervised learning. The transfer curves are optimized based on the classification of global or local image content. It showed that it is possible to obtain the desired effect by learning from other computationally expensive enhancement algorithms or expert-tuned examples through the trained approach. Looking back, the thesis reveals a single versatile framework for video enhancement applications. It widens the application scope by including new content classifiers and new processing models and offers scalabilities with solutions to reduce the number of classes, which can greatly accelerate the algorithm design

The finite element method in low speed aerodynamics

The finite element procedure is shown to be of significant impact in design of the 'computational wind tunnel' for low speed aerodynamics. The uniformity of the mathematical differential equation description, for viscous and/or inviscid, multi-dimensional subsonic flows about practical aerodynamic system configurations, is utilized to establish the general form of the finite element algorithm. Numerical results for inviscid flow analysis, as well as viscous boundary layer, parabolic, and full Navier Stokes flow descriptions verify the capabilities and overall versatility of the fundamental algorithm for aerodynamics. The proven mathematical basis, coupled with the distinct user-orientation features of the computer program embodiment, indicate near-term evolution of a highly useful analytical design tool to support computational configuration studies in low speed aerodynamics