A Survey on Metric Learning for Feature Vectors and Structured Data

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new method

Structure Preserving and Scalable Simulation of Colliding Systems

Predictive computational tools to study granular materials are important in fields ranging from the geosciences and civil engineering to computer graphics. The simulation of granular materials, however, presents many challenges. The behavior of a granular medium is fundamentally multi-scale, with pair-wise interactions between discrete granules able to influence the continuum-scale evolution of a bulk material. Computational techniques for studying granular materials must therefore contend with this multi-scale nature. This research first addresses both the question of how to accurately model interactions between grains and the question of how to achieve multi-scale simulations of granular materials. We propose a novel rigid body contact model and a time integration technique that, for the first time, are able to simultaneously capture five key features of rigid body impact. We further validate this new model and time integration method by reproducing computationally challenging phenomena from granular physics. We next propose a technique to couple discrete and continuum models of granular materials to one another. This hybrid model reveals a family of possible discretizations suitable for simulation. We derive an explicit integration technique from this framework that is able to capture phenomena previously reserved for discrete treatments, including frictional jamming, while treating bulk regions of the material with a continuum model. To effectively handle the large plastic deformations inherent in the evolution of a granular medium, we further propose a method to dynamically update which regions are treated with a discrete model and which regions are treated with a continuum model. We demonstrate that hybrid simulations of a dynamically evolving granular material are possible and practical, and lay the foundation for further algorithmic development in this space. Finally, as the the tools used in computational science and engineering become progressively more complex, the ability to effectively train students in the field becomes increasingly important. We address the question of how to train students from a computer science background in numerical computation techniques by proposing a new system to automatically vet and identify problems in numerical simulations. This system has been deployed at the undergraduate and graduate level in a course on physical simulation at Columbia University, and has increased both student retention and student satisfaction with the course

Loss Scaling and Step Size in Deep Learning Optimizatio

Deep learning training consumes ever-increasing time and resources, and that isdue to the complexity of the model, the number of updates taken to reach goodresults, and both the amount and dimensionality of the data. In this dissertation,we will focus on making the process of training more efficient by focusing on thestep size to reduce the number of computations for parameters in each update.We achieved our objective in two new ways: we use loss scaling as a proxy forthe learning rate, and we use learnable layer-wise optimizers. Although our workis perhaps not the first to point to the equivalence of loss scaling and learningrate in deep learning optimization, ours is the first to leveraging this relationshiptowards more efficient training. We did not only use it in simple gradient descent,but also we were able to extend it to other adaptive algorithms. Finally, we usemetalearning to shed light on relevant aspects, including learnable lossesand optimizers. In this regard, we developed a novel learnable optimizer andeffectively utilized it to acquire an adaptive rescaling factor and learning rate,resulting in a significant reduction in required memory during training

Computer-assisted animation creation techniques for hair animation and shade, highlight, and shadow

制度:新 ; 報告番号:甲3062号 ; 学位の種類:博士(工学) ; 授与年月日:2010/2/25 ; 早大学位記番号:新532

Learning by correlation for computer vision applications: from Kernel methods to deep learning

Learning to spot analogies and differences within/across visual categories is an arguably powerful approach in machine learning and pattern recognition which is directly inspired by human cognition. In this thesis, we investigate a variety of approaches which are primarily driven by correlation and tackle several computer vision applications

Sensor Signal and Information Processing II

In the current age of information explosion, newly invented technological sensors and software are now tightly integrated with our everyday lives. Many sensor processing algorithms have incorporated some forms of computational intelligence as part of their core framework in problem solving. These algorithms have the capacity to generalize and discover knowledge for themselves and learn new information whenever unseen data are captured. The primary aim of sensor processing is to develop techniques to interpret, understand, and act on information contained in the data. The interest of this book is in developing intelligent signal processing in order to pave the way for smart sensors. This involves mathematical advancement of nonlinear signal processing theory and its applications that extend far beyond traditional techniques. It bridges the boundary between theory and application, developing novel theoretically inspired methodologies targeting both longstanding and emergent signal processing applications. The topic ranges from phishing detection to integration of terrestrial laser scanning, and from fault diagnosis to bio-inspiring filtering. The book will appeal to established practitioners, along with researchers and students in the emerging field of smart sensors processing