Physically inspired methods and development of data-driven predictive systems

Traditionally building of predictive models is perceived as a combination of both science and art. Although the designer of a predictive system effectively follows a prescribed procedure, his domain knowledge as well as expertise and intuition in the field of machine learning are often irreplaceable. However, in many practical situations it is possible to build well–performing predictive systems by following a rigorous methodology and offsetting not only the lack of domain knowledge but also partial lack of expertise and intuition, by computational power. The generalised predictive model development cycle discussed in this thesis is an example of such methodology, which despite being computationally expensive, has been successfully applied to real–world problems. The proposed predictive system design cycle is a purely data–driven approach. The quality of data used to build the system is thus of crucial importance. In practice however, the data is rarely perfect. Common problems include missing values, high dimensionality or very limited amount of labelled exemplars. In order to address these issues, this work investigated and exploited inspirations coming from physics. The novel use of well–established physical models in the form of potential fields, has resulted in derivation of a comprehensive Electrostatic Field Classification Framework for supervised and semi–supervised learning from incomplete data. Although the computational power constantly becomes cheaper and more accessible, it is not infinite. Therefore efficient techniques able to exploit finite amount of predictive information content of the data and limit the computational requirements of the resource–hungry predictive system design procedure are very desirable. In designing such techniques this work once again investigated and exploited inspirations coming from physics. By using an analogy with a set of interacting particles and the resulting Information Theoretic Learning framework, the Density Preserving Sampling technique has been derived. This technique acts as a computationally efficient alternative for cross–validation, which fits well within the proposed methodology. All methods derived in this thesis have been thoroughly tested on a number of benchmark datasets. The proposed generalised predictive model design cycle has been successfully applied to two real–world environmental problems, in which a comparative study of Density Preserving Sampling and cross–validation has also been performed confirming great potential of the proposed methods.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Face Recognition Using Correlation Filters

最小平均相關能量濾波器很常應用於影像辨識系統。當用來訓練濾波器的影像經過適當的選擇，我們可以得到滿意的辨識率。然而我們知道最小平均相關能量濾波器的效能對於失真非常敏感，因此我們利用滿足某些特定的條件來最佳化濾波器的效能、而不使用嚴格的限制。為了避免多於不必要的影像來訓練濾波器，而浪費記憶體和運算量，在這篇論文我們提供一個演算法可以自動從資料庫裡找尋最適合拿來訓練濾波器的影像。另外基於安全和隱私因素，我們必須把加密功能加入我們的濾波器設計中，避免資料被盜用的可能，然後我們會證明使用的加密功能並不會影響到辨識能力。最後為了處理資料量龐大的情形跟人臉的非線性變形，我們使用種類相關的特徵分析跟使用更高維的資料訊息來做相關性。整合這些技術，我們將濾波器改良的更可以克服現實生活當中可能產生的問題。The minimum average correlation energy (MACE) filter is a well known correlation filter for pattern recognition. The recognition rates will be attractive while choosing the training images properly. But the MACE filter is sensitive to the distortion, thus we optimize the filter by removing the hard constraint and satisfying certain criterion, which is so-called unconstrained correlation filter. In order to avoid redundant training images, here we provide an algorithm to automatically choose the proper training images from a dataset. For security issue, we also use encryption method in the filter design and we will show that the encryption process do not affect the recognition performance. Finally for the large scale database and the nonlinear distortion in human face, we improve the recognition performance by using class-dependent feature analysis and correntropy function, where correntropy is a positive definite function that generalizes the concept of correlation by utilizing higher order moment information of signal structure. Using these technologies, we can make the filter more practical in real application.Chapter 1 Introduction 1 Chapter 2 Minimum Average Correlation Energy Filter 5 2.1 Introduction 5 2.2 Notation 5 2.3 Problem Definition 6 2.4 Solution 8 2.5 Properties of MACE Filter 10 2.6 The Fitness Metric of MACE Filter 13 2.7 Experiments 13 2.8 Conclusion 20 Chapter 3 Unconstrained Correlation Filter 21 3.1 Introduction 21 3.2 Notation 21 3.3 Derivation of the Filter Equation 22 3.3.1 Unconstrained Minimum Average Correlation Energy (MACE) Filter 23 3.3.2 Maximum Average Correlation Height (MACH) Filter 24 3.3.3 Generalized MACH (GMACH) Filter 30 3.4 Relation Between Maximum Average Correlation Height, minimum-Squared-Error Synthetic Discriminant Function, and Minimum Average Correlation Energy Filter 31 3.5 Optimal Trade-Off Design of Minimum-Average Correlation Height Filter 33 3.6 Amplitude-Normalized Filter Design 34 3.7 Experiments 36 3.8 Conclusion 38 Chapter 4 Face recognition on a large scale database with cancelable correlation filter 39 4.1 Introduction 39 4.2 Background Knowledge of Correlation Filter 39 4.3 How to Choose the Proper Training Set 40 4.3.1 Algorithm 41 4.3.2 Simulation Result 42 4.4 Cancelable Correlation Filter 44 4.4.1 Structure of the Cancelable Correlation Filter 44 4.4.2 PSR invariance to arbitrary convolution kernels 47 4.4.3 Experiment and Simulation 49 4.5 Redundant Class-dependence Feature Analysis 49 4.6 Kernel Correlation Filters 55 4.7 Conclusion 57 Chapter 5 The Correntropy MACE Filter 59 5.1 Introduction 59 5.2 Notation 60 5.3 Derivation of the Correntropy MACE Filter 60 5.4 CFA using Correntropy MACE Filter 63 5.5 Conclusion 64 Chapter 6 Conclusions and Future Work 65 Reference 6